summaryrefslogtreecommitdiff
path: root/test/py/u_boot_console_sandbox.py
Commit message (Collapse)AuthorAgeLines
* test/py: pass -v option when executing sandboxStephen Warren2016-04-11-0/+1
| | | | | | | | | This shows more output, such as the internal output generated by the unit test ("ut") command, which makes it easier to debug issues. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Tom Rini <trini@konsulko.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: support running sandbox under gdbserverStephen Warren2016-02-08-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement command--line option --gdbserver COMM, which does two things: a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel. b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it. This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is: Window 1: ./test/py/test.py --bd sandbox --gdbserver localhost:1234 Window 2: gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234' When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed. Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because: a) gdb's output would then be processed by the test script, and likely confuse it causing false failures. b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb. While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters. c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication. An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too. --gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: run sandbox in source directoryStephen Warren2016-01-28-1/+1
| | | | | | | | Some unit tests expect the cwd of the sandbox process to be the root of the source tree. Ensure that requirement is met. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: pass test DTB to sandboxStephen Warren2016-01-28-1/+6
| | | | | | | This is required for at least "ut dm" to operate correctly. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: use " for docstringsStephen Warren2016-01-28-10/+10
| | | | | | | | | | | Python's coding style docs indicate to use " not ' for docstrings. test/py has other violations of the coding style docs, since the docs specify a stranger style than I would expect, but nobody has complained about those yet:-) Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: move U-Boot respawn trigger to the test coreStephen Warren2016-01-28-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, U-Boot was lazilly (re-)spawned if/when a test attempted to interact with it, and no active connection existed. This approach was simple, yet had the disadvantage that U-Boot might be spawned in the middle of a test function, e.g. after the test had already performed actions such as creating data files, etc. In that case, this could cause the log to contain the sequence (1) some test logs, (2) U-Boot's boot process, (3) the rest of that test's logs. This isn't optimally readable. This issue will affect the upcoming DFU and enhanced UMS tests. This change converts u_boot_console to be a function-scoped fixture, so that pytest attempts to re-create the object for each test invocation. This allows the fixture factory function to ensure that U-Boot is spawned prior to every test. In practice, the same object is returned each time so there is essentially no additional overhead due to this change. This allows us to remove: - The explicit ensure_spawned() call from test_sleep, since the core now ensures that the spawn happens before the test code is executed. - The laxy calls to ensure_spawned() in the u_boot_console_* implementations. The one downside is that test_env's "state_ttest_env" fixture must be converted to a function-scoped fixture too, since a module-scoped fixture cannot use a function-scoped fixture. To avoid overhead, we use the same trick of returning the same object each time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Implement pytest infrastructureStephen Warren2016-01-20-0/+79
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are: - Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect. - There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C. - It is reasonably simple to interact with U-Boot in this way. A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too. The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks. See README.md for more details! Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Tested-by: Michal Simek <michal.simek@xilinx.com> Tested-by: Simon Glass <sjg@chromium.org> Acked-by: Simon Glass <sjg@chromium.org> #v3