Validation, Error Handling and Unit Tests


Home » Courses » Building Network Automation Solutions » Validation, Error Handling and Unit Tests

You have to be registered for this online course, or have this course selected as part of an active Expert Subscription to access all course materials.

Validation, Error Handling and Unit Tests

The fifth module of the network automation online course covers testing, debugging, error handling, and continuous integration.

More information...

Understanding Ansible

Have you completed the Ansible Deep Dive module of the Ansible for Networking Engineers online course? If haven't, please do it now, or you'll have problems understanding the material discussed in this module.

56:11 Validation, Error Handling and Unit Tests

In this section you’ll learn how to:

  • Write fail-safe playbooks
  • Recover from Ansible module failures and report them
  • Use Ansible logging and debugging tools
  • Debug individual components of your Ansible playbooks (tasks, plays or modules/plugins)
  • Build and automate unit tests
  • Validate input data and device state
Debugging, Testing and Input Data Validation 2.9M 2017-02-07
Writing Fail-Safe Playbooks 15:25 2017-02-08
Debugging, Logging and Testing 23:23 2017-02-08
Input Data Validation 17:23 2017-02-08

Real-Life Examples

Testing Data Models by Donald Johnson

56:15 Network Continuous Integration

In his Network Continuous Integration presentation Pete Lumbis explained the basics of Continuous Integration and Continuous Delivery (CI/CD) concepts and demonstrated how you can use GitLab and virtual labs to test network device configurations prior to production deployment.

Network CI-CD 2.5M 2017-02-07
Change Management Today and Tomorrow 9:05 2017-02-08
Continuous Integration with GitLab 24:40 2017-02-08
NetDevOps Toolbox 22:30 2017-02-08

1:10:17 Continuous Integration with GitLab CI

GitLab CI is one of the commonly-used networking-focused continuous integration tools due to its agent-based architecture.

In this section Pete Lumbis explained how he uses GitLab CI to test his network automation scripts and device configurations.

Continuous Integration Concepts 24:03 2017-11-08
GitLab CI 22:36 2017-11-08
GitLab Demo 23:38 2017-11-08
Slide deck 837K 2017-11-04

2:21:03 Testing Network Automation Systems

Kristian Larsson described a testing methodology based on the approach his team is using to test the network automation system they're building for Deutsche Telekom Terastream project.

He started with unit and system tests, described various approaches to device mocking, explained how to do end-to-end system tests, and shown how you can use vrnetlab (an open source virtual network lab environment focused on testing use cases) in Continuous Integration pipeline.

In the second part of his presentation he described how you could test correctness of BGP configurations with docker-based BGP feeds, and concluded with a discussion of robustness and feature flags.

Testing Overview 16:02 2019-04-10
Unit and System Tests 18:46 2019-04-10
End-to-End Tests 12:36 2019-04-10
vrnetlab 24:42 2019-04-10
Using vrnetlab 27:54 2019-04-10
Testing BGP 18:23 2019-04-10
Robustness, Safety and Feature Flags 22:40 2019-04-10
Slide Deck 19M 2019-04-09

Hands-on exercises

You’ll add data validation and error handling to the service deployment playbook you created during Week 4, and create unit tests to test your new code with a variety of invalid inputs.

Homework: Add logging, testing and validation to your solution 3.7K 2018-12-24
Submit your homework
Overview: Submitting Hands-On Exercise Solutions

Optional Self-Study Materials

1:13:31 Building a Testing Pipeline

When you automate, you want to make sure that you are not automating bugs and mistakes, thereby making them spread across your environment. That is why you want to have a proper testing pipeline in place!

In his presentation, Gabriele Gerbino gives some practical examples building a simple testing pipeline by using Travis-CI and VMs/containers to make sure that you can sleep better once you hit the DEPLOY red button.

Why and What Should You Test 19:18 2018-10-18
Build the Testing Infrastructure 29:59 2018-10-18
Execute the Tests 17:45 2018-10-18
Homework Assignments 6:29 2018-10-18
Slide deck 1.7M 2018-10-19

1:11:39 Writing Scalable and Reliable Software

David Barroso explained how to design and write an application focusing on three aspects; scalability, maintainability, and reliability.

He described design principles, tools, and patterns you can use to grow your applications without compromising long-term quality, and increase confidence and speed when developing new features or changing existing ones.

In the second half of the presentation, David illustrated these principles with a simple application that generates interface- and BGP configuration from a network data model.

Introduction 3:50 2021-03-03
Developer Tools 7:01 2021-03-03
Static Typing with Mypy 12:48 2021-03-03
Coding Example 29:35 2021-03-03
Testing 15:48 2021-03-03
Summary 2:37 2021-03-03
Slide Deck 3.6M 2020-10-29

Additional resources

Explore validation with NAPALM, test-driven development and distributed testing tools. You might also be interested in tools that make your Ansible development less error-prone.

Validating deployments with NAPALM

Validating deployments with NAPALM blog post
napalm-validate section in the Ansible course
Test-Driven Network Development with Michael Kashin
Distributed On-Demand Network Testing with Matt Oswalt


YAML validator
Ansible lint (best-practices checker)
Automated Ansible playbook/rules review
Structured Git pre-commit hooks

Other interesting resources

Unit testing Ansible modules
You started this section on %started% Mark completed