Embedded Expertise

Become a partner >>
Embedded Expertise logo

Cukinia: Powerful Simplicity for Linux Embedded Firmware Testing

For developers working on Linux-based embedded firmware, ensuring code quality and system reliability is crucial. Cukinia emerges as a valuable tool in this process, offering a simple yet powerful framework for running basic system-level validation tests directly on the target system, eliminating the need for a separate test bench.

What is Cukinia used for?

Cukinia is particularly suited for developers working on:

  • Linux-based embedded firmware for various devices, ranging from simple peripherals to complex systems.
  • Automating validation testing directly on the target device, as opposed to a test bench. This expedites the test setup and allows for testing the firmware in its intended operational environment.
  • Integrating a user-friendly and integrable testing tool into their existing development workflows, particularly those involving build suites like Buildroot or Yocto. Cukinia integrates seamlessly with these frameworks.

Cukinia: Authored by Savoir-Faire Linux

Cukinia is a creation of Savoir-Faire Linux, a Canadian company with a strong presence in France, specializing in providing expert Linux services and solutions. It is distributed under the terms of the flexible Apache license.

Benefits of Cukinia

There are several advantages to using Cukinia:

  • Simplicity: Cukinia is designed to be accessible even for developers without prior experience in automated testing. Its intuitive interface and clear documentation make it easy to learn and use.
  • Power Under the Hood: Don’t be fooled by its simplicity! Cukinia offers a wide range of testing functionalities, enabling comprehensive functional, performance, and regression testing.
  • Fast: Cukinia’s focus on essential tests makes it fast to execute, removing the burden of running heavy, detailed test suites that require a separate test bench. This allows for frequent testing during the development cycle.
  • Minimal Footprint: Cukinia is a lightweight framework built primarily on Bash scripts. Since most systems already have Bash pre-installed, adding Cukinia requires minimal additional software and resources.
  • Integration with Build Suites: Cukinia integrates seamlessly with firmware generation frameworks like Buildroot and Yocto, making it easy to incorporate into your existing build process.

Continuous Integration with Development

  • Due to its speed and ease of use, Cukinia is ideal for running every time a new firmware image is generated during development. This allows for catching regressions or bugs early in the development cycle.
  • As you add new features to your firmware, it’s crucial to update your Cukinia test scripts as well. These updates serve two purposes:
    • Testing proper integration of the new feature: New tests ensure the new feature functions as intended and interacts correctly with existing functionalities.
    • Regression prevention: Updated tests help to identify any unintended consequences of the new feature that might cause regressions in existing functionalities. This helps to maintain overall code stability.

Test Examples

Limitations of Cukinia

To get a feel for Cukinia, here’s a basic test description example that verifies some networking settings:


as "Network interface eth0 is present" cukinia_test -L "/sys/class/net/eth0"
cukinia_systemd_unit sshd.socket
cukinia_listen4 tcp 22

Cukinia provides clear and concise results for each executed test. A successful test displays in green, while a failing test displays in red. Additionally, detailed information about the cause of the test failure is provided. Here is a successful result of the above test:


[PASS] Network interface eth0 is present
[PASS] Checking if systemd unit "sshd.socket" is active
[PASS] Checking tcp v4 port "22" is open

If the sshd server is stopped, the tests fails with the following report:


[PASS] Network interface eth0 is present
[FAIL] Checking if systemd unit "sshd.socket" is active
[FAIL] Checking tcp v4 port "22" is open

It’s important to remember that Cukinia focuses on basic system validation testing. For more complex testing scenarios or those requiring in-depth environment simulation, other testing tools might be better suited.

Cukinia features for powerful simplicity

Cukinia goes beyond a basic system testing framework by offering robust capabilities that streamline the development process for Linux-based embedded firmware. Here’s a non-exhaustive list of its functionalities:

- Organized Testing with Independent Suites

  • Modular Approach: Cukinia allows you to structure your tests into independent suites. This promotes organization by grouping tests based on specific functionalities or system aspects they target.
  • Run All, Run Some: The ability to execute entire test suites or select specific ones provides flexibility. You can choose to run a comprehensive test suite covering various aspects or a targeted suite focusing on a particular feature.
  • Example Use Cases:
    • One suite can parse system logs for errors and warnings, ensuring system stability.
    • Another suite can verify the presence and functionality of specific devices connected to the system.

- Rich Built-in Test Features

Cukinia provides a wide range of pre-built test functionalities, eliminating the need to write complex scripts from scratch. These built-in features allow you to perform various validation tasks efficiently:

  • Kernel configuration: Easily inspect the status of specific kernel configuration options. With Cukinia, you can quickly determine if an option is enabled, disabled, or loaded as a module.
  • Log Analysis: Walk through system logs and search for specific keywords or patterns. This helps identify potential issues or errors flagged during system operation.
  • File and Device Verification: Easily check the presence or absence of specific files or devices on the system. This ensures that crucial components are present and functional.
  • Special File Validation: Verify special files like block devices or character devices to ensure their proper configuration and operation.
  • Permission Checks: Confirm that files and directories have the appropriate permissions assigned, crucial for system security.
  • Symlink Targets: Validate the targets of symbolic links, ensuring they point to the intended locations.
  • Process Monitoring: Check for the presence of specific processes running on the system. This helps verify that essential services are functioning as expected.
  • systemd Service Status: Investigate the status of systemd services, including whether they are enabled, running, or failed. This ensures critical system services are operational.
  • … and more.
 
 

- Beyond Built-in Features

  • Cukinia caters to advanced testing scenarios as well:

    • Scripted Tests: You can write custom Python scripts to define more complex test logic, enabling you to tailor tests to specific needs.
    • Custom Test Primitives: The framework allows you to create your own reusable test primitives, extending its built-in functionalities.

- Clear and Concise Test Results

Cukinia provides a user-friendly output format for test results, making it easy to interpret the test status.

Each test result is displayed clearly as either PASS (green) or FAIL (red), offering a quick visual indication of the test outcome. The reports are also easy to parse with generic tools like grep, sed or awk.

Cukinia and the Development Workflow

In a previous article, we discussed the importance of building firmware with separate targets: one optimized for deployment and another with additional debugging features. The debug target, with its expanded functionalities, serves as the ideal platform for running Cukinia tests during development. This allows developers to thoroughly validate their code without impacting the final deployment image.

While installing Cukinia in the deployment image might be feasible, it’s generally not recommended. The primary purpose of the deployment image is to provide a lean and efficient system for the end user. Including Cukinia would add unnecessary overhead. However, for specific scenarios where remote field diagnostics are crucial, a minimal Cukinia installation within the deployment image could be considered.

Wrap-up

Cukinia is a valuable asset for Linux embedded firmware developers aiming to enhance the quality and reliability of their systems. Its simplicity, power, flexibility, and fast execution make it an ideal choice for running frequent validation tests directly on the target device during development. By integrating Cukinia with your build suite and development workflows, you can ensure the smooth operation of final products while catching regressions and bugs early in the development cycle.

Feel free to explore the official Cukinia repository to gain a deeper understanding and begin integrating it into your development workflows.

The development staff at Embedded Expertise are daily users of Cukinia. Feel free to contact us to guide your through your testing journey.

Cukinia logo