summaryrefslogtreecommitdiff
path: root/test/py
Commit message (Collapse)AuthorAgeLines
* tests: py: fix NameError exception if bdi cmd is not supportedHeiko Schocher2016-05-13-0/+1
| | | | | | | | | | | | test/py raises an error, if a board has not enabled bdi command > pytest.skip('bdinfo command not supported') E NameError: global name 'pytest' is not defined import pytest in test/py/u_boot_utils.py fixes this. Signed-off-by: Heiko Schocher <hs@denx.de> Reviewed-by: Stephen Warren <swarren@nvidia.com>
* test/py: dfu: wait for USB device to go away at bootStephen Warren2016-05-06-0/+2
| | | | | | | | | | | | | It can take a while for a host machine to notice that a USB device has disconnected, and process the change. At the end of the DFU test, we wait up to 10 seconds for this to happen. This change makes the test wait the same (up to) 10 seconds at the start of the test for any previously active USB device-mode session to be cleaned up. Such as session might have been used to download U-Boot into memory for example; this is certainly true on my Tegra test systems. This changes should solve the DFU test intermittency issues I've been seeing on some Tegra devices. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* tests: py: dfu: Provide functionality to set test and dummy files alt settingsLukasz Majewski2016-04-25-5/+21
| | | | | | | | | | | | | | | | | | | | After concatenation of "dfu_alt_info" variable from "dfu_alt_boot" and "dfu_alt_system" it may happen that test and dummy files alt settings are different than default 0 and 1. This patch provides the ability to set different values for them. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - replace variables declarations with ones read from configuration file - remove not necessary str() conversion at DFU host command generation Changes for v2: - generate "alt_info" automatically - use file names as alt settings instead of numerical values - extend in-code documentation
* tests: py: dfu: Add functionality to set different u-boot's dfu env variableLukasz Majewski2016-04-25-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default (on almost all systems) the dfu env variable, which defines available alt settings, is named as "dfu_alt_info". However on some platforms (i.e. Odroid XU3), the 'dfu_alt_info' is concatenated from other variables - namely 'dfu_alt_boot' and 'dfu_alt_system' at run time (when one types 'dfu 0 mmc 0' for first time). 'dfu_alt_boot' describes alt settings which depend on boot medium - for example boot loader's LBA sectors which are different on eMMC and SD card because of e.g. MBR/GPT. 'dfu_alt_system' describes board agnostic alt settings - like rootfs, kernel. On such system we can only append/modify this env variable. Because of the above, we must have way to modify other than "dfu_ale_info" variable to perform tests. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - None Changes for v2: - Rewrite of "alt_info_env_name" variable description - Use of get() method on python's dictionary to easily obtain default value
* tests: py: dfu: Add variables to store dfu alt numbers for test and dummy filesLukasz Majewski2016-04-25-4/+8
| | | | | | | | | | | | | This patch replaces hardcoded (i.e. 0 and 1) values passed to dfu_{read|write} with variables. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - Replace per module global variables with ones defined inside a function Changes for v2: - None
* test/py: README: link to example hook scriptsStephen Warren2016-04-11-0/+6
| | | | | | | | | | When implementing test/py hook scripts, it's helpful to read some working examples. Provide a link to some. The link was mentioned in the commit message which first added test/py, but not in any documentation file. Suggested-by: Lukasz Majewski <l.majewski@samsung.com> Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Tom Rini <trini@konsulko.com>
* test/py: Add support for loading image via tftp to specified locationMichal Simek2016-04-11-1/+5
| | | | | | | | | | | | | | For example this setting: env__net_tftp_readable_file = { "fn": "ep108/image.ub", "addr": 0x10000000, "size": 25846296, "crc32": "b726f9de", } Signed-off-by: Michal Simek <michal.simek@xilinx.com> Acked-by: Stephen Warren <swarren@nvidia.com>
* test/py: pass -v option when executing sandboxStephen Warren2016-04-11-0/+1
| | | | | | | | | This shows more output, such as the internal output generated by the unit test ("ut") command, which makes it easier to debug issues. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Tom Rini <trini@konsulko.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: skip tests that require large CONFIG_SYS_MAXARGSStephen Warren2016-02-26-0/+8
| | | | | | | | | | | | | | | | | test_hush_if_test.py executes commands that require large values of CONFIG_SYS_MAXARGS. Detect cases where the configured value is too low and skip those tests. Ideally, this would be implemented inside console.run_command(). However, the command passed to that function is already a completely formed string, and determining its argument count usage would require splitting commands at ;, handling quoting to deal with arguments containing spaces, etc. Even passing the command as a list wouldn't solve all these issues, since we'd still need to split commands on ; and deal with cases like "if test ..." which consumes 0 of the argument count. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: use space to interrupt autobootStephen Warren2016-02-26-1/+1
| | | | | | | | | | | | | | | Sending CTRL-C to QEMU's stdin aborts the process, even if stdin is being used as a serial port (at least in the raspi2 machine with "qemu -serial stdin"). Avoid sending CTRL-C to U-Boot to prevent it exiting. I'd originally used CTRL-C to make sure that if the character used to abort autoboot ended up being treated as part of a command as well, it'd abort command entry and return the prompt to a known state. However, this is not needed, since aborting the autoboot eats the character used to do that. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Add option to skip SPL signature checkingMichal Simek2016-02-25-1/+3
| | | | | | | | | | | | | | Provide user option to skip SPL signature verification for cases where u-boot is build with SPL support but full U-Boot is also verified without SPL. If you want to support this feature please add env__spl_skipped = True to your boardenv configuration file. For example Xilinx Zynq is using this feature where the same U-Boot binary is checked with SPL and without SPL(with FSBL). Signed-off-by: Michal Simek <michal.simek@xilinx.com>
* test/py: only check for SPL signature if SPL uses serial outputHeiko Schocher2016-02-24-2/+7
| | | | | | | | | check for U-Boot SPL signature only if SPL really has a serial output. So check if CONFIG_SPL_SERIAL_SUPPORT is active in board config. Signed-off-by: Heiko Schocher <hs@denx.de> Tested-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Stephen Warren <swarren@nvidia.com>
* test/py: put "Starting U-Boot" into separate log sectionStephen Warren2016-02-15-1/+3
| | | | | | | | | | | | | | | | | The initial boot of U-Boot happens within the context of the first test that needs to access the U-Boot console when there is no existing connection. This keeps all activity nestled within test execution, which fits well into the pytest model. However, this mingles the U-Boot startup logs with the execution of some test(s), which hides find the boundary between the two. To solve this, wrap the "Starting U-Boot" logic into a separate log section. If the user wishes, they can simply collapse this log section when viewing the HTML log, to concentrate purely on the test's own interaction. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: handle exceptions in console creationStephen Warren2016-02-15-7/+15
| | | | | | | | | | | | | | | | | | | | | | | | u_boot_console.exec_attach.get_spawn() performs two steps: 1) Spawn a process to communicate with the serial console. 2) Reset the board so that U-Boot starts running from scratch. Currently, if an exception happens in step (2), no cleanup is performed on the process created in step (1). That process stays running and may e.g. hold serial port locks, or simply continue to read data from the serial port, thus preventing it from reaching any other process that attempts to read from the same serial port later. While there is error cleanup code in u_boot_console_base.ensure_spawned(), this is not triggered since the exception prevents assignment to self.p there, and hence the exception handler has no object to operate upon in cleanup_spawn(). Solve this by enhancing u_boot_console.exec_attach.get_spawn() to clean up any objects it has created. In theory, u_boot_spawn.Spawn's constructor has a similar issue, so fix this too. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: print summary in test orderStephen Warren2016-02-15-8/+8
| | | | | | | | | | | | Use lists rather than sets to record the status of tests. This causes the test summary in the HTML file to be generated in the same order as the tests are (or would have been) run. This makes it easier to locate the first failed test. The log for this test might have interesting first clues re: interaction with the environment (e.g. hardware flashing, serial console, ...) and may help tracking down external issues. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix CONFIG_SPL testStephen Warren2016-02-15-1/+1
| | | | | | | | | | The Python ini file parser that's used to parse .config converts all keys to lower-case. Hence, all queries against the results must use lower-case. Fix u_boot_console.ensure_spawned() to test CONFIG_SPL correctly, or the connection will fail for boards that have SPL. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: don't import pexpectStephen Warren2016-02-15-1/+0
| | | | | | | | The code replaced pexpect with custom code long ago. Don't import the unused module so it doesn't need to be installed. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add docs for gdbserver and pytest optionsStephen Warren2016-02-15-4/+56
| | | | | | | | Add documentation describing the new --gdbserver feature, and some common pytest options. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: run all "ut" subtestsStephen Warren2016-02-15-24/+108
| | | | | | | | | | | Invoke each "ut"-based unit test as a separate pytest. Now that the DM unit test runs under test/py, remove the manual shell script that invokes it. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org> Tested-by: Simon Glass <sjg@chromium.org> # v2, on sandbox
* test/py: capture the entire U-Boot version at bootStephen Warren2016-02-09-8/+3
| | | | | | | | | The existing regex simply ensures that the captured version string doesn't go past the end of a line. We really want to grab as much as possible. Do this by explicitly including a ) character at the end of the regex to match the last character of the version test. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: fix off-by-one error in spawn matching codeStephen Warren2016-02-09-1/+1
| | | | | | | | A regex match object's .end() value is already the index after the match, not the index of the last character in the match, so there's no need to add 1 to point past the match. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: HTML awesome!Stephen Warren2016-02-09-78/+178
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement three improvements to the HTML log file: - Ability to expand/contract sections. All passing sections are contracted at file load time so the user can concentrate on issues requiring action. - The overall status report is copied to the top of the log for easy access. - Add links from the status report to the test logs, for easy navigation. This all relies on Javascript and the jquery library. If the user doesn't have Javascript enabled, or jquery can't be downloaded, the log should look and behave identically to how it did before this patch. A few notes on the diff: - A few more 'with log.section("xxx")' were added, so that all stream blocks are kept within a section block for consistent HTML entity nesting structure. This changed indentation in a few places, making the diff look slightly larger. - HTML entity IDs are cleaned up. We assign simple incrementing integer IDs now, rather than using mangled test names which were possibly invalid. - Sections and streams now use common CSS class names (in addition to the current separate class names) to more easily share the new behaviour. This also reduces the CSS file size since rules don't need to be duplicated. - An "OK" status is logged after some external command executions so that make and flash steps are auto-contracted at log file load time, assuming they passed. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: exit(1) if there are problems running py.testStephen Warren2016-02-09-0/+1
| | | | | | | | | | The test/py/test.py wrapper script catches exceptions thrown when exec()ing py.test in order to print a helpful error message. However, the exception handling code squashes the exception and so the script exits with a non-zero exit code, leading callers to believe that it passed. Fix this. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: support running sandbox under gdbserverStephen Warren2016-02-08-9/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement command--line option --gdbserver COMM, which does two things: a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel. b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it. This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is: Window 1: ./test/py/test.py --bd sandbox --gdbserver localhost:1234 Window 2: gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234' When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed. Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because: a) gdb's output would then be processed by the test script, and likely confuse it causing false failures. b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb. While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters. c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication. An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too. --gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: dfu: allow boardenv to specify test sizesStephen Warren2016-01-28-2/+10
| | | | | | | | | | | | | Allow the env__dfu_configs boardenv data to specify the set of DFU transfer sizes to test. Manually specifying test sizes is useful if you wish to test multiple DFU configurations (e.g. SD card ext4 filesystem, SD card whole raw partition, RAM, etc.), but don't want to test every single transfer size on each, to avoid bloating the overall time taken by testing. If the boardenv doesn't specify a set of sizes, the built-in list is used as a default, preserving backwards-compatibility. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix a couple typos in commentsStephen Warren2016-01-28-2/+2
| | | | | | | s/updata/update/. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: run sandbox in source directoryStephen Warren2016-01-28-3/+7
| | | | | | | | Some unit tests expect the cwd of the sandbox process to be the root of the source tree. Ensure that requirement is met. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: pass test DTB to sandboxStephen Warren2016-01-28-1/+6
| | | | | | | This is required for at least "ut dm" to operate correctly. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: correctly log xfail/xpass testsStephen Warren2016-01-28-24/+74
| | | | | | | | | | Tests can complete in passed, skipped, xpass, xfailed, or failed, states. Currently the U-Boot log generation code doesn't handle the xfailed or xpass states since they aren't used. Add support for the remaining states. Without this, tests that xfail end up being reported as skipped. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: detect another "bad pattern" in console outputStephen Warren2016-01-28-0/+2
| | | | | | | | | | | | Many error situations in U-Boot print the message: ### ERROR ### Please RESET the board ### Add this to the list of bad patterns the test system detects. One practical advantage of this change is to detect the case where sandbox is told to use a particular DTB file, and the file cannot be opened. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: check for bad patterns everywhere we waitStephen Warren2016-01-28-7/+19
| | | | | | | | | | | | | | | | Currently, bad patterns are only honored when executing a shell command. Other cases, such as the initial boot-up of U-Boot or when interacting with command output rather than gathering all output prior to the shell prompt, do not currently look for bad patterns in console output. This patch makes sure that bad patterns are honored everywhere. One benefit of this change is that if U-Boot sandbox fails to start up, the error message it emits can be caught immediately, rather than relying on a (long) timeout when waiting for the expected signon message and/or command prompt. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test.py: calculate bad patterns on change onlyStephen Warren2016-01-28-24/+25
| | | | | | | | | | A future patch will use the bad_patterns array in multiple places. Rather than duplicating the code to calculate it, or even sharing it in a function and simply calling it redundantly when nothing has changed, only re-calculate the list when some change is made to it. This reduces work. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix spawn.expect multiple match handlingStephen Warren2016-01-28-1/+1
| | | | | | | | | | | Multiple patterns may be passed to spawn.expect(). The pattern which matches at the earliest position should be designated as the match. This aspect works correctly. When multiple patterns match at the same position, priority should be given the the earliest entry in the list of patterns. This aspect does not work correctly. This patch fixes it. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Provide custom IDs when parametrizing testsStephen Warren2016-01-28-1/+13
| | | | | | | | | | | | | | | | | | | | | When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as: dfu[env__usb_dev_port0-env__dfu_config0] dfu[env__usb_dev_port0-env__dfu_config1] Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as: dfu[micro_b-emmc] dfu[devport2-ram] If the boardenv file doesn't define custom names, the code falls back to the old algorithm. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: Quote consistencyStephen Warren2016-01-28-42/+42
| | | | | | | | | | | When converting test/py from " to ', I missed a few places (or added a few inconsistencies later). Fix these. Note that only quotes in code are converted; double-quotes in comments and HTML are left as-is, since English and HTML use " not '. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: use " for docstringsStephen Warren2016-01-28-245/+245
| | | | | | | | | | | Python's coding style docs indicate to use " not ' for docstrings. test/py has other violations of the coding style docs, since the docs specify a stranger style than I would expect, but nobody has complained about those yet:-) Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: make net test aware of USB and PCI enumerationStephen Warren2016-01-28-13/+15
| | | | | | | | | | | | | | | | | | | | | | The existing net test executes a list of commands supplied by boardenv variable env__net_pre_commands. The idea was that boardenv would know whether the Ethernet device was attached to USB, PCI, ... and hence was the best place to put any commands required to probe the device. However, this approach doesn't scale well when attempting to use a single boardenv across multiple branches of U-Boot, some of which require "pci enum" to enumerate PCI and others of which don't, or don't /yet/ simply because various upstream changes haven't been merged down. This patch updates the test to require that the boardenv state which HW features are required for Ethernet to work, and lets the test itself map that knowledge to the set of commands to execute. Since this mapping is part of the test script, which is part of the U-Boot code/branch, this approach is more scalable. It also feels cleaner, since again boardenv is only providing data, rather than test logic. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: dfu: error out if USB device already existsStephen Warren2016-01-28-0/+6
| | | | | | | | | | | | | | | | | | The DFU test requests U-Boot configure its USB controller in device mode, then waits for the host machine to enumerate the USB device and create a device node for it. However, this wait can be fooled if the USB device node already exists before the test starts, e.g. if some previous software stack already configured the USB controller into device mode and never de-configured it. This "previous software stack" could even be another test/py test, if U-Boot's own USB teardown does not operate correctly. If this happens, dfu-util may be run before U-Boot is ready to serve DFU commands, which may cause false test failures. Enhance the dfu test to fail if the device node exists before it is expected to. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: make crash detection more robustStephen Warren2016-01-28-13/+7
| | | | | | | | | | | | | | | | | | | | test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset. Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset. Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add a networking testStephen Warren2016-01-28-0/+153
| | | | | | | | | | | This tests: - dhcp (if indicated by boardenv file). - Static IP network setup (if provided by boardenv file). - Ping. - TFTP get. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move find_ram_base() into u_boot_utilsStephen Warren2016-01-28-39/+41
| | | | | | | | find_ram_base() is a shared utility function, not a core part of the U-Boot console interaction. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add DFU testStephen Warren2016-01-28-0/+262
| | | | | | | | | | | | | | | Add a test of DFU functionality to the Python test suite. The test starts DFU in U-Boot, waits for USB device enumeration on the host, executes dfu-util multiple times to test various transfer sizes, many of which trigger USB driver edge cases, and finally aborts the DFU command in U-Boot. This test mirrors the functionality previously available via the shell scripts in test/dfu, and hence those are removed too. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: ums: add filesystem-based testingStephen Warren2016-01-28-48/+188
| | | | | | | | | | | | | | | | | | | | | Enhance the UMS test to optionally mount a partition and read/write a file to it, validating that the content written and read back are identical. This enhancement is backwards-compatible; old boardenv contents that don't define the new configuration data will cause the test code to perform as before. test/ums/ is deleted since the Python test now performs the same testing that it did. The code is also re-written to make use of the recently added utility module, and split it up into nested functions so the overall logic of the test process can be followed more easily without the details cluttering the code. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add various utility codeStephen Warren2016-01-28-0/+190
| | | | | | | | Add various common utility functions. These will be used by a forthcoming re-written UMS test, and a brand-new DFU test. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: optionally ignore errors from shell commandsStephen Warren2016-01-28-2/+6
| | | | | | | | | | Sometimes it's useful to run shell commands and ignore any errors. One example might be cleanup logic; if a test-case experiences an error, the cleanup logic might experience an error too, and we don't want that error to mask the original error, so we want to ignore the subsequent error. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: log when tests send CTRL-CStephen Warren2016-01-28-0/+1
| | | | | | | | | | Write a note to the log file when a test sends CTRL-C to U-Boot. This makes it easier to follow what's happening in the logs, especially since U-Boot doesn't echo the character back to its output, so there's no other signal of what's going on. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: drain console log at the end of any failed testStephen Warren2016-01-28-0/+39
| | | | | | | | | | | | | | | | Tests may fail for a number of reasons, and in particular for reasons other than a timeout waiting for U-Boot to print expected data. If the last operation that a failed test performs is not waiting for U-Boot to print something, then any trailing output from U-Boot during that test's operation will not be logged as part of that test, but rather either along with the next test, or even thrown away, potentiall hiding clues re: the test failure reason. Solve this by explicitly draining (and hence logging) the U-Boot output in the case of failed tests. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move U-Boot respawn trigger to the test coreStephen Warren2016-01-28-12/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, U-Boot was lazilly (re-)spawned if/when a test attempted to interact with it, and no active connection existed. This approach was simple, yet had the disadvantage that U-Boot might be spawned in the middle of a test function, e.g. after the test had already performed actions such as creating data files, etc. In that case, this could cause the log to contain the sequence (1) some test logs, (2) U-Boot's boot process, (3) the rest of that test's logs. This isn't optimally readable. This issue will affect the upcoming DFU and enhanced UMS tests. This change converts u_boot_console to be a function-scoped fixture, so that pytest attempts to re-create the object for each test invocation. This allows the fixture factory function to ensure that U-Boot is spawned prior to every test. In practice, the same object is returned each time so there is essentially no additional overhead due to this change. This allows us to remove: - The explicit ensure_spawned() call from test_sleep, since the core now ensures that the spawn happens before the test code is executed. - The laxy calls to ensure_spawned() in the u_boot_console_* implementations. The one downside is that test_env's "state_ttest_env" fixture must be converted to a function-scoped fixture too, since a module-scoped fixture cannot use a function-scoped fixture. To avoid overhead, we use the same trick of returning the same object each time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix timeout to be absoluteStephen Warren2016-01-28-1/+6
| | | | | | | | | | | | | | Currently, Spawn.expect() imposes its timeout solely upon receipt of new data, not on its overall operation. In theory, this could cause the timeout not to fire if U-Boot continually generated output that did not match the expected patterns. Fix the code to additionally impose a timeout on overall operation, which is the intended mode of operation. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add a test for the sleep commandStephen Warren2016-01-20-0/+24
| | | | | | | | Execute "sleep", and validate that it sleeps for approximately the correct amount of time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
generated by cgit v1.1 at 2024-12-25 18:14:52 +0000