Data Models
Home » Courses » Building Network Automation Solutions » Data Models
Data Models
The third module of the network automation online course describes data models and data stores. We'll start with an introduction, figure out what NETCONF, YANG and OpenConfig are, and finish with a data model abstraction case study.
1:53:10 Data Models and Data Model Transformation |
||
This section introduces data models and helps you answer these questions:
|
||
Data Model Introduction | 17:55 | 2021-04-21 |
Data Model Introduction QA | 8:53 | 2021-04-21 |
Device-Level Data Models | 15:33 | 2021-04-21 |
Optimizing a Device-Level Data Model | 17:53 | 2021-04-21 |
Infrastructure and Services Data Models | 11:40 | 2021-04-21 |
Data Model Transformations | 17:45 | 2021-04-21 |
Data Model Transformation in Automation Workflow | 23:31 | 2021-08-28 |
Slide Deck | 3.3M | 2021-01-20 |
Further Reading – Data Models |
||
Network Automation Data Model Optimization | ||
Data Model Transformation Concepts | ||
Data Transformation Example (Jinja2 / Ansible / Makefile) | ||
Complex Data Transformation Example: Lab Topology Building Tool | ||
Source Code for the Lab Topology Building Tool | ||
Further Reading – Data Transformation |
||
Data Model Transformations in Network Automation Solutions | ||
Worth Reading: Data Manipulation in Jinja2 | ||
1:20:31 Formatting, Describing, and Storing Data |
||
After you built your data model, you have to create data structures (device, link, network, service, or customer data) in a format readable by humans and machines, describe the data model for documentation and validation purposes, and store the data somewhere. This section dives deep into presentation formats (XML, JSON, YAML), data description languages (including YANG), and data stores, from text files and Excel to NoSQL databases. |
||
Data Representation | 18:56 | 2021-08-28 |
Data Model Descriptions | 15:32 | 2021-08-28 |
YANG and OpenConfig | 13:17 | 2021-08-28 |
Data Stores | 18:56 | 2021-08-28 |
Selecting a Data Store | 13:50 | 2021-08-28 |
Further Reading – Data Representation |
||
Beware XML-to-JSON Information Loss (Junos with Ansible) | ||
XML-to-JSON Information Loss, Cisco Nexus OS Edition | ||
Fixing XML-to-JSON Conversion Challenges | ||
Further Reading – Data Validation |
||
Data Validation with JSON Schema | ||
Simple Data Validation with YANG Using yanglint | ||
Interesting Tool: Schema Enforcer | ||
New Ansible Data Validation Module(s) | ||
Automation Should Prevent Operator Errors | ||
Validating Data in GitOps-Based Automation | ||
Automation Solution: Testing Data Models | ||
Further Reading – Data Stores |
||
Text Files or Relational Database? | ||
Using YAML Instead of Excel in Network Automation Solutions | ||
Growing Beyond Ansible host_vars and group_vars | ||
What’s the Big Deal with Validation? | ||
Other Interesting Blog Posts |
||
What’s the Big Deal with Validation? | ||
What Is Continuous Integration? | ||
Continuous Integration in Network Automation | ||
Firewall Ruleset Automation with CI Pipeline | ||
From Excel to Network Infrastructure as Code with Carl Buchmann | ||
Sample YANG Data Models |
||
YANG Module Classification (RFC 8199) | ||
Service models explained (RFC 8309) | ||
Customer-focused YANG model for L3VPN service delivery | ||
Network-centric L3VPN YANG Model | ||
YANG model for L2VPN service delivery | ||
37:05 Single Source of Truth |
||
After mastering the data model-, model transformation-, and data store concepts, you're ready for one of the most important network automation topics: single source of truth. |
||
Network Automation Source-of-Truth | 17:00 | 2021-12-17 |
Building a Source-of-Truth | 20:05 | 2021-12-17 |
Slide Deck | 783K | 2021-11-01 |
Related Blog Posts |
||
Building the Network Automation Source of Truth | ||
Building Network Automation Source-of-Truth (Part 2) | ||
Creating Automation Source-of-Truth from Device Configurations | ||
Device Configurations Are Not a Good Source of Truth | ||
Read Network Device Information with REST API and Store It Into a Database | ||
Building Automation Device Inventory with Open Source Tools | ||
Big Red Button for Network Automation | ||
Further Reading |
||
Git as a Source of Truth for Network Automation | ||
Hierarchical Configuration Up and Running | ||
1:19:59 NETCONF, YANG and OpenConfig |
||
The guest presentation by Marcel Wiget (Juniper) describes YANG data models, briefly introduces NETCONF, and shows how you can use OpenConfig or your custom data model (described in YANG) to configure Junos devices. |
||
Vendor-Neutral YANG Data Models | 1.8M | 2017-01-20 |
Introduction to NETCONF and YANG | 26:27 | 2017-01-27 |
OpenConfig and Custom YANG Data Models | 14:24 | 2017-01-27 |
Using YANG Data Models on Junos | 39:08 | 2017-01-27 |
58:07 Using YANG Data Models on Traditional Network Devices with napalm-yang |
||
While configuring network devices with OpenConfig sounds really interesting, don't expect to be able to do it any time soon - the vendors are (as usual) very reluctant to add yet another layer of indirection on top of their already-bloated software. David Barroso decided to fix that problem within NAPALM, and wrote a set of libraries (napalm-yang) that perform translation between OpenConfig (or any other) YANG data model and device configurations. |
||
What Are YANG and OpenConfig | 15:32 | 2017-10-13 |
napalm-yang Basics | 10:29 | 2017-10-13 |
Advanced napalm-yang Features | 10:01 | 2017-10-13 |
napalm-yang Integration with Ansible | 22:05 | 2017-10-13 |
Slide deck: OpenConfig with NAPALM | 2.7M | 2017-10-09 |
Understanding Ansible |
||
You won’t be able to follow the rest of this module without a thorough understanding of Ansible. If you haven’t completed the Ansible for Networking Engineers online course, please do it now, and focus on the data modeling part of DMVPN case study. |
||
42:29 Abstract Everything |
||
In this part of the Network Automation Use Cases webinar David Barroso walks you through layers of network automation abstraction: from abstracting vendor-specific APIs with NAPALM to abstracting device configuration differences, and finally abstracting device state into infrastructure data model. |
||
Template Multi-Vendor Deployments | 8:52 | 2017-07-21 |
Abstract Vendor API | 8:42 | 2017-07-21 |
Automate Everything | 6:35 | 2017-07-21 |
Abstract Everything | 18:20 | 2017-07-21 |
Additional resources |
||
Slide deck | 4.1M | 2016-11-18 |
Source code for examples used in this section | ||
1:28:48 Intent-Based Networking |
||
Most of the intent-based systems are nothing more than a fancy orchestration system with an abstraction layer. This section describes the many levels of abstraction you can implement in such a system. |
||
What Is Intent-Based Networking | 19:00 | 2022-04-14 |
Device Configuration as Intent | 7:29 | 2022-04-14 |
Abstracting and Managing Device-Level Intent | 10:18 | 2022-04-14 |
Replacing Intent Definition | 6:38 | 2022-04-14 |
Network-Wide Intent | 6:06 | 2022-04-14 |
Automated Remediation | 12:42 | 2022-04-14 |
Back to Reality | 16:29 | 2022-04-14 |
Questions and Answers | 10:06 | 2022-04-14 |
Additional resources |
||
Slide deck | 2.3M | 2022-03-16 |
Intent-Based Networking-related Blog Posts | ||
Further Reading |
||
Intent-Based Networking Taxonomy | ||
The Maslow’s Pyramid of Needs of Intent-Based Networking by Benoît Claise | ||
How Facebook Learned to Stop Worrying and Love the Network | ||
Circular Dependencies Considered Harmful (World-Wide Facebook Outage) | ||
Automation Solution: Deploy BGP Routing with YANG Data Models | ||
Using Flow Tracking to Build Firewall Rulesets... and the Halting Problem | ||
The Law of Leaky Abstractions (Joel on Software) | ||
The ABC of Vendor Lock-In | ||
Shift Network Spend From Premium Products to Premium People | ||
Service-Oriented IETF Data Models |
||
Service models explained (RFC 8309) | ||
YANG model for L3VPN service delivery | ||
YANG model for L2VPN service delivery | ||
34:02 Transforming Data Models with Ansible |
||
Orchestration systems with an abstraction layer usually use network- and services data models to describe the desired system functionality, and device (or node) data models to describe the target state of the system. In every such system, someone has to perform the mapping between the high-level and low-level data models, and this section describes how you can do that with Ansible. |
||
Transforming Data Models with Ansible | 17:28 | 2017-10-13 |
Data Models - Discussion Questions | 16:34 | 2017-10-13 |
Slide Deck: Transforming Data Models with Ansible | 2.3M | 2017-10-20 |
1:23:27 Free items Data Center Fabrics Data Model |
||
In another network automation use case Dinesh Dutt talked about automated data center fabric deployments. Focus on how simple his data model is and how he builds a whole fabric from just a few configurable parameters. |
||
Challenges of Data Center Fabric Deployments
![]() |
20:02 | 2017-01-24 |
Separate Data from Code
![]() |
12:56 | 2017-01-24 |
Ansible Tips and Tricks
![]() |
9:19 | 2017-01-24 |
Validation
![]() |
12:51 | 2017-01-24 |
Staging and Production
![]() |
11:10 | 2017-01-24 |
Fabric Deployment Demo
![]() |
17:09 | 2017-01-24 |
Additional resources |
||
Slide deck | 1.6M | 2017-01-24 |
Source code for examples used in this section | ||
Free items Hands-on exercises |
||
In the hands-on exercises you’ll build a data model for your network automation solution and store it in one or more YAML files. |
||
Homework: Create an infrastructure or services data model | 6.4K | 2018-12-24 |
Submit your homework | ||
Overview: Submitting Hands-On Exercise Solutions | ||
Useful Tips and Tricks |
||
Data (de)Duplication in Network Automation Data Models | ||
Data Munging with Ansible and Jinja2
![]() |
7.5K | 2018-12-24 |
Optional Materials |
||
1:18:10 Manipulating Network Information with Python |
||
Hans Verkerk guided us you through some easy-to-understand scripts written in Python. These scripts abstract Cisco IOS configurations into Python data structures, enabling you to query the data gathered from network devices using standard Python utilities in a simple but effective way. |
||
Storing Information into Python Data Structures | 32:18 | 2020-02-23 |
Parsing Device Configuration Files | 10:17 | 2020-02-23 |
Validating Campus Network Configurations | 12:59 | 2020-02-23 |
Improved Device Configuration Parsing | 5:10 | 2020-02-23 |
Migration Example | 17:26 | 2020-02-23 |
Slide Deck | 1.7M | 2019-10-25 |
Additional Informations |
||
Data Manipulation with Python Built-In Superheroes | ||
Source code for live demos | ||
Additional resources |
||
Want to know even more about data models in network automation? You might enjoy these webinars and presentations. |
||
NETCONF and YANG | 4:25:00 | |
If you want to know more about NETCONF, watch at least the first part of NETCONF and YANG webinar with David Gee (~ 1 hour if you skip the hands-on part). You might also want to watch the NETCONF Deep Dive section to understand the NETCONF protocol details. |
||
The Single Source of Truth for Automatic Network Configuration by Andy Davidson | ||
Slides for "The Single Source of Truth for Automatic Network Configuration" presentation |