test/py: Implement pytest infrastructure

This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-Boot console, run the entire script of tests against it, and
summarize the results. Advantages of this approach are:

- Testing is performed in the same way a user or script would interact
  with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
  It is asserted that writing test-related code in Python is simpler and
  more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.

A few simple tests are provided as examples. Soon, we should convert as
many as possible of the other tests in test/* and test/cmd_ut.c too.

The hook scripts, relay control utilities, and udev rules I use for my
own HW setup are published at https://github.com/swarren/uboot-test-hooks.

See README.md for more details!

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org> #v3

1 files changed, 65 insertions, 0 deletions

diff --git a/test/py/u_boot_console_exec_attach.py b/test/py/u_boot_console_exec_attach.py
new file mode 100644
index 0000000..0ca9e7c
--- /dev/null
+++ b/test/py/u_boot_console_exec_attach.py
@@ -0,0 +1,65 @@
+# Copyright (c) 2015 Stephen Warren
+# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Logic to interact with U-Boot running on real hardware, typically via a
+# physical serial port.
+
+import sys
+from u_boot_spawn import Spawn
+from u_boot_console_base import ConsoleBase
+
+class ConsoleExecAttach(ConsoleBase):
+    '''Represents a physical connection to a U-Boot console, typically via a
+    serial port. This implementation executes a sub-process to attach to the
+    console, expecting that the stdin/out of the sub-process will be forwarded
+    to/from the physical hardware. This approach isolates the test infra-
+    structure from the user-/installation-specific details of how to
+    communicate with, and the identity of, serial ports etc.'''
+
+    def __init__(self, log, config):
+        '''Initialize a U-Boot console connection.
+
+        Args:
+            log: A multiplexed_log.Logfile instance.
+            config: A "configuration" object as defined in conftest.py.
+
+        Returns:
+            Nothing.
+        '''
+
+        # The max_fifo_fill value might need tweaking per-board/-SoC?
+        # 1 would be safe anywhere, but is very slow (a pexpect issue?).
+        # 16 is a common FIFO size.
+        # HW flow control would mean this could be infinite.
+        super(ConsoleExecAttach, self).__init__(log, config, max_fifo_fill=16)
+
+        self.log.action('Flashing U-Boot')
+        cmd = ['u-boot-test-flash', config.board_type, config.board_identity]
+        runner = self.log.get_runner(cmd[0], sys.stdout)
+        runner.run(cmd)
+        runner.close()
+
+    def get_spawn(self):
+        '''Connect to a fresh U-Boot instance.
+
+        The target board is reset, so that U-Boot begins running from scratch.
+
+        Args:
+            None.
+
+        Returns:
+            A u_boot_spawn.Spawn object that is attached to U-Boot.
+        '''
+
+        args = [self.config.board_type, self.config.board_identity]
+        s = Spawn(['u-boot-test-console'] + args)
+
+        self.log.action('Resetting board')
+        cmd = ['u-boot-test-reset'] + args
+        runner = self.log.get_runner(cmd[0], sys.stdout)
+        runner.run(cmd)
+        runner.close()
+
+        return s