Cuica

Test Suites
Test SuitesSpecifications

Specifications

Test Suite Specifications

Description

Test Suite Specifications are documents written in either YAML or JSON, which describe the Test Suite parameters, the Test Cases, and the values to assign to bundled Test Scripts and Test Binaries (if applicable).

This is both a serializable and deserializable data structure. Any read-only metadata returned by the Cuica is ignored if resubmitted. Examples of read-only metadata are creation time and UUIDs assigned by the system.

Minimal Specification

The following are the V1.0 minimal specifications:

Test Suite Specification (YAML)
"Minimal Test Suite":
  schemaVersion: 1.0
Test Suite Specification (JSON)
{
    "Minimal Test Suite": {
        "schemaVersion": "1.0"
    }
}

Full Specification

The version 1.0 specification follows:

Test Suite Specification (YAML)
# Label of the test suite used for printing; any unicode string
# (optional, defaults to the first segment of the id field)
"YAML Reference Test Suite":
  # Verion of the test suite schema; must be in [1.0..2.0)
  # Represents which parser to use
  schemaVersion: "1.0"
 
  # Verion of the test suite as defined by the user
  version: v3.2.1
 
  # Information about who created or owns the test suite (optional)
  author: John Doe
 
  # Description of the test suite; any unicode string
  # (optional, defaults to "")
  description: This is a description from the YAML
 
  # The deployment config to use for all cases; if there is no deployment 
  # to be done, omit it. For any missing field, the default is used
  deployment:
    deployerType: openocd
    deployerConfig: Cuica
    dutConfig: rp2040
 
  # Tags for easier filtering and organization of tests
  #(optional, defaults to [])
  tags: 
    - tag1
    - tag2
 
  # An optional set of binaries to import from bundle
  # Note that order isn't preserved by location but by priority and 
  # the dependencies
  addBinaries: 
    my_binary: # The label of the binary, any unicode string
      # The file name of the binary as relative to the top of the
      # archive; any unicode string
      file: my_binary.elf
 
      # The version of the binary; any unicode string; optional
      # defaults to ""
      version: 2b88c978
 
    my_binary2:
      file: my_binary2.elf
      version: 2b88c978
 
 
  # An optional array of scripts to import from bundle
  addScripts:
    # The label of the script, any unicode string; must be unique
    # and will replace any existing matching the label already
    # in the bank
    my_script:
      # The filename of the setup as relative to the top of
      # the archive; any unicode string; exclusive field of "python"
      file: my_script.py
 
      # The version of the script; any unicode string;
      # optional defaults to ""
      version: 2b88c978
 
    my_script2:
      # Inline script; exclusive field of "file"
      data: |
        print('I am also some python')
        print('Multiple line')
 
      version: 2b88c978
 
 
  # An array of at least one test case reference;
  # The test cases will be run in order listed below
  testCases: 
   # A reference to a previously imported binary by label
    - binary:
        label: my_binary
 
      # A reference to a previously imported setup script by label
      setupScript:
        label: my_script
 
      # A reference to a previously imported script by label
      testScript:
        label: my_script2
 
      # Expected output (artifact) format;
      #valid choices in [json, binary, string]; optional
      resultsFormat: json
 
  # Metadata about this test suite; ignored as an input and generated as 
  # an output when exporting
  metadata:
    # Test test suite UUID
    testSuiteUuid: 179ce39c-596e-11ee-9f64-f7bb10631fb0
      
    # The Uuid of the request which added it
    addRequestUuid: cf63ac18-596e-11ee-a5f0-479e5ab78339
 
    # The original filename of the bundle which was used to create this
    # test suite
    originalFilename: this_test_suite.tar
 
    # The time when the test suite was first imported
    # (YYYY-MM-DDThh:mm:ss±hh:mm format)
    creationTime: "2023-09-19T14:30:45+02:00"
 
    # The last time the test suite was modified 
    # (YYYY-MM-DDThh:mm:ss±hh:mm format)
    modificationTime: "2023-09-19T14:30:45+02:00"
 
    # The time when the test suite was processed and ready to run
    # (YYYY-MM-DDThh:mm:ss±hh:mm format)
    availableTime: "2023-09-19T14:30:45+02:00"
Test Suite Specification (JSON*)
{
    "JSON Reference Test Suite": {
        "schemaVersion": "1.0",
        "version": "v4.3.2",
        "author": "Jane Doe",
        "description": "This is a description from the JSON",
        "deployment": {
            "deployerType": "openocd",
            "deployerConfig": "Cuica",
            "dutConfig": "rp2040"
        },
        "tags": [
            "tag1",
            "tag2"
        ],
        "addBinaries": {
            "my_binary": {
                "file": "my_binary.elf",
                "version": "2b88c978"
            }
        },
        "addScripts": {
            "my_script": {
                "format": "python",
                "file": "test_script.py",
                "data": "print('I am some python')",
                "version": "2b88c978"
            }
        },
        "testCases": [
            {
                "binary": {
                    "label": "my_binary"
                },
                "setupScript": {
                    "label":  "my_script"
                },
                "testScript": {
                    "label": "my_script"
                },
                "resultsFormat": "json"
            }
        ]
    }
}
*Refer to comments in YAML specification
Copyright © Uatha LLC. All rights reserved.