summaryrefslogtreecommitdiff
path: root/test/py/tests
Commit message (Collapse)AuthorAgeLines
* test: Adjust run_command_list() to return a list of stringsSimon Glass2016-08-05-1/+1
| | | | | | | | Return one string for each command that was executed. This seems cleaner. Suggested-by: Teddy Reed <teddy.reed@gmail.com> Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Stephen Warren <swarren@nvidia.com>
* test: Add a function to restart U-BootSimon Glass2016-08-05-2/+3
| | | | | | | | | | | Add a proper function for this rather than using internal functions. Use it in the single call site. Also, do a restart at the end of the vboot test to reset to the normal device tree. Signed-off-by: Simon Glass <sjg@chromium.org> Suggested-by: Stephen Warren <swarren@nvidia.com>
* test: vboot: Put each test variant in its own sectionSimon Glass2016-08-05-5/+5
| | | | | | | | | Use 'cons.log.section' feature to split up the test output. This makes it easier to read. Suggested-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Simon Glass <sjg@chromium.org>
* test: Rename sha to sha_algo and pass it aroundSimon Glass2016-08-05-25/+28
| | | | | | | | | | Rename this argument and pass it to each function that needs it, instead of making it global. Suggested-by: Stephen Warren <swarren@nvidia.com> Suggested-by: Teddy Reed <teddy.reed@gmail.com> Signed-off-by: Simon Glass <sjg@chromium.org>
* test: Drop the cmd() functionSimon Glass2016-08-05-10/+12
| | | | | | | | | Instead of this, use the existing run_and_log() function, enhanced to support a command string as well as a list of arguments. Suggested-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Simon Glass <sjg@chromium.org>
* test: Fix typos in commentsSimon Glass2016-08-05-6/+6
| | | | | | | | Fix some typos in various files introduced with the vboot test conversion. Reported-by: Teddy Reed <teddy.reed@gmail.com> Signed-off-by: Simon Glass <sjg@chromium.org>
* test: Fix typos and tidy upSimon Glass2016-08-05-2/+4
| | | | | | | | | | Fix review comments that were missed at the time. Also explain why we need to regenerate the device tree for each test. Reported-by: Teddy Reed <teddy.reed@gmail.com> Suggested-by: Stephen Warren <swarren@nvidia.com> Signed-off-by: Simon Glass <sjg@chromium.org> Fixes: f6349c3c (test: Add a README)
* test/py: vboot can be run only at SandboxMichal Simek2016-07-22-0/+1
| | | | | | | | | | | | | | | Getting this error: Zynq> sb load hostfs - 100 /home/monstr/data/disk/u-boot/build-zynq_zc706/test.fit Unknown command 'sb' - try 'help' because sb command is present only for Sandbox obj-$(CONFIG_SANDBOX) += host.o that's why mark this test to be run only at Sandbox Signed-off-by: Michal Simek <michal.simek@xilinx.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: use absolute dts path in vboot testStephen Warren2016-07-22-1/+1
| | | | | | | | | Without this, the test fails if the test is run with a cwd other than the root of the U-Boot source tree. Fixes: 8729d582595d ("test: Convert the vboot test to test/py") Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test: Adjust the of-platdata test run conditionSimon Glass2016-07-16-1/+1
| | | | | | | This should be spl_of_platdata, since otherwise it will try to run on boards that don't support of-platdata. Signed-off-by: Simon Glass <sjg@chromium.org>
* Merge git://git.denx.de/u-boot-dmTom Rini2016-07-15-0/+42
|\
| * dm: Add a test for of-platdataSimon Glass2016-07-14-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a simple test which checks that the of-platdata system is working correctly. The sequence is as follows: - SPL starts up and probes all the UCLASS_MISC drivers - There are 3 of these in sandbox.dts - Therefore there should be 3 U_BOOT_DEVICE() declarations in dt-platdata.c - These should produce 3 sandbox_spl_test devices - Each device prints out its platform data when probed - This test checks for this output and compares it against expectations Signed-off-by: Simon Glass <sjg@chromium.org>
* | test: Convert the vboot test to test/pySimon Glass2016-07-14-0/+376
|/ | | | | | | | Now that we have a suitable test framework we should move all tests into it. The vboot test is a suitable candidate. Rewrite it in Python and move the data files into an appropriate directory. Signed-off-by: Simon Glass <sjg@chromium.org>
* test/py: fix CONFIG_ testsStephen Warren2016-07-08-5/+5
| | | | | | | | | | | Some CONFIG_ variables were recently renamed, but test/py wasn't updated to match. This causes some tests to be skipped. Fix test/py so the tests are run. Fixes: 11636258981a ("Rename reset to sysreset") Fixes: f1f9d4fac527 ("hush: complete renaming CONFIG_SYS_HUSH_PARSER to CONFIG_HUSH_PARSER") Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: fix printenv signon message disable codeStephen Warren2016-06-19-1/+2
| | | | | | | | | | | CONFIG_VERSION_VARIABLE isn't always defined, so we can't simply look up its value directly, or an exception will occur if it isn't defined. Instead, we must use .get() to supply a default value if the variable isn't defined. Fixes: da37f006e7c5 ("tests: py: disable main_signon check for printenv cmd") Acked-by: Heiko Schocher <hs@denx.de> Signed-off-by: Stephen Warren <swarren@nvidia.com>
* tests: py: disable main_signon check for printenv cmdHeiko Schocher2016-06-09-1/+5
| | | | | | | | | | | | if CONFIG_VERSION_VARIABLE is set, the U-Boot environment contains a "vers" variable with the current U-Boot version string. If now "printenv" is called, test/py fails as it detects the main_sign string, which is in this case correct. So check only the main_sign as an error, if CONFIG_VERSION_VARIABLE is not set. Signed-off-by: Heiko Schocher <hs@denx.de>
* test/py: dfu: wait for USB device to go away at bootStephen Warren2016-05-06-0/+2
| | | | | | | | | | | | | It can take a while for a host machine to notice that a USB device has disconnected, and process the change. At the end of the DFU test, we wait up to 10 seconds for this to happen. This change makes the test wait the same (up to) 10 seconds at the start of the test for any previously active USB device-mode session to be cleaned up. Such as session might have been used to download U-Boot into memory for example; this is certainly true on my Tegra test systems. This changes should solve the DFU test intermittency issues I've been seeing on some Tegra devices. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* tests: py: dfu: Provide functionality to set test and dummy files alt settingsLukasz Majewski2016-04-25-5/+21
| | | | | | | | | | | | | | | | | | | | After concatenation of "dfu_alt_info" variable from "dfu_alt_boot" and "dfu_alt_system" it may happen that test and dummy files alt settings are different than default 0 and 1. This patch provides the ability to set different values for them. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - replace variables declarations with ones read from configuration file - remove not necessary str() conversion at DFU host command generation Changes for v2: - generate "alt_info" automatically - use file names as alt settings instead of numerical values - extend in-code documentation
* tests: py: dfu: Add functionality to set different u-boot's dfu env variableLukasz Majewski2016-04-25-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default (on almost all systems) the dfu env variable, which defines available alt settings, is named as "dfu_alt_info". However on some platforms (i.e. Odroid XU3), the 'dfu_alt_info' is concatenated from other variables - namely 'dfu_alt_boot' and 'dfu_alt_system' at run time (when one types 'dfu 0 mmc 0' for first time). 'dfu_alt_boot' describes alt settings which depend on boot medium - for example boot loader's LBA sectors which are different on eMMC and SD card because of e.g. MBR/GPT. 'dfu_alt_system' describes board agnostic alt settings - like rootfs, kernel. On such system we can only append/modify this env variable. Because of the above, we must have way to modify other than "dfu_ale_info" variable to perform tests. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - None Changes for v2: - Rewrite of "alt_info_env_name" variable description - Use of get() method on python's dictionary to easily obtain default value
* tests: py: dfu: Add variables to store dfu alt numbers for test and dummy filesLukasz Majewski2016-04-25-4/+8
| | | | | | | | | | | | | This patch replaces hardcoded (i.e. 0 and 1) values passed to dfu_{read|write} with variables. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Stephen Warren <swarren@nvidia.com> --- Changes for v3: - Replace per module global variables with ones defined inside a function Changes for v2: - None
* test/py: Add support for loading image via tftp to specified locationMichal Simek2016-04-11-1/+5
| | | | | | | | | | | | | | For example this setting: env__net_tftp_readable_file = { "fn": "ep108/image.ub", "addr": 0x10000000, "size": 25846296, "crc32": "b726f9de", } Signed-off-by: Michal Simek <michal.simek@xilinx.com> Acked-by: Stephen Warren <swarren@nvidia.com>
* test/py: skip tests that require large CONFIG_SYS_MAXARGSStephen Warren2016-02-26-0/+8
| | | | | | | | | | | | | | | | | test_hush_if_test.py executes commands that require large values of CONFIG_SYS_MAXARGS. Detect cases where the configured value is too low and skip those tests. Ideally, this would be implemented inside console.run_command(). However, the command passed to that function is already a completely formed string, and determining its argument count usage would require splitting commands at ;, handling quoting to deal with arguments containing spaces, etc. Even passing the command as a list wouldn't solve all these issues, since we'd still need to split commands on ; and deal with cases like "if test ..." which consumes 0 of the argument count. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: run all "ut" subtestsStephen Warren2016-02-15-0/+29
| | | | | | | | | | | Invoke each "ut"-based unit test as a separate pytest. Now that the DM unit test runs under test/py, remove the manual shell script that invokes it. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org> Tested-by: Simon Glass <sjg@chromium.org> # v2, on sandbox
* test/py: support running sandbox under gdbserverStephen Warren2016-02-08-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement command--line option --gdbserver COMM, which does two things: a) Run the sandbox process under gdbserver, using COMM as gdbserver's communication channel. b) Disables all timeouts, so that if U-Boot is halted under the debugger, tests don't fail. If the user gives up in the middle of a debugging session, they can simply CTRL-C the test script to abort it. This allows easy debugging of test failures without having to manually re-create the failure conditions. Usage is: Window 1: ./test/py/test.py --bd sandbox --gdbserver localhost:1234 Window 2: gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234' When using this option, it likely makes sense to use pytest's -k option to limit the set of tests that are executed. Simply running U-Boot directly under gdb (rather than gdbserver) was also considered. However, this was rejected because: a) gdb's output would then be processed by the test script, and likely confuse it causing false failures. b) pytest by default hides stdout from tests, which would prevent the user from interacting with gdb. While gdb can be told to redirect the debugee's stdio to a separate PTY, this would appear to leave gdb's stdio directed at the test scripts and the debugee's stdio directed elsewhere, which is the opposite of the desired effect. Perhaps some complicated PTY muxing and process hierarchy could invert this. However, the current scheme is simple to implement and use, so it doesn't seem worth complicating matters. c) Using gdbserver allows arbitrary debuggers to be used, even those with a GUI. If the test scripts invoked the debugger themselves, they'd have to know how to execute arbitary applications. While the user could hide this all in a wrapper script, this feels like extra complication. An interesting future idea might be a --gdb-screen option, which could spawn both U-Boot and gdb separately, and spawn the screen into a newly created window under screen. Similar options could be envisaged for creating a new xterm/... too. --gdbserver currently only supports sandbox, and not real hardware. That's primarily because the test hooks are responsible for all aspects of hardware control, so there's nothing for the test scripts themselves can do to enable gdbserver on real hardware. We might consider introducing a separate --disable-timeouts option to support use of debuggers on real hardware, and having --gdbserver imply that option. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* test/py: dfu: allow boardenv to specify test sizesStephen Warren2016-01-28-2/+10
| | | | | | | | | | | | | Allow the env__dfu_configs boardenv data to specify the set of DFU transfer sizes to test. Manually specifying test sizes is useful if you wish to test multiple DFU configurations (e.g. SD card ext4 filesystem, SD card whole raw partition, RAM, etc.), but don't want to test every single transfer size on each, to avoid bloating the overall time taken by testing. If the boardenv doesn't specify a set of sizes, the built-in list is used as a default, preserving backwards-compatibility. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix a couple typos in commentsStephen Warren2016-01-28-2/+2
| | | | | | | s/updata/update/. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: Provide custom IDs when parametrizing testsStephen Warren2016-01-28-0/+6
| | | | | | | | | | | | | | | | | | | | | When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as: dfu[env__usb_dev_port0-env__dfu_config0] dfu[env__usb_dev_port0-env__dfu_config1] Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as: dfu[micro_b-emmc] dfu[devport2-ram] If the boardenv file doesn't define custom names, the code falls back to the old algorithm. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: Quote consistencyStephen Warren2016-01-28-4/+4
| | | | | | | | | | | When converting test/py from " to ', I missed a few places (or added a few inconsistencies later). Fix these. Note that only quotes in code are converted; double-quotes in comments and HTML are left as-is, since English and HTML use " not '. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: use " for docstringsStephen Warren2016-01-28-89/+89
| | | | | | | | | | | Python's coding style docs indicate to use " not ' for docstrings. test/py has other violations of the coding style docs, since the docs specify a stranger style than I would expect, but nobody has complained about those yet:-) Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: make net test aware of USB and PCI enumerationStephen Warren2016-01-28-13/+15
| | | | | | | | | | | | | | | | | | | | | | The existing net test executes a list of commands supplied by boardenv variable env__net_pre_commands. The idea was that boardenv would know whether the Ethernet device was attached to USB, PCI, ... and hence was the best place to put any commands required to probe the device. However, this approach doesn't scale well when attempting to use a single boardenv across multiple branches of U-Boot, some of which require "pci enum" to enumerate PCI and others of which don't, or don't /yet/ simply because various upstream changes haven't been merged down. This patch updates the test to require that the boardenv state which HW features are required for Ethernet to work, and lets the test itself map that knowledge to the set of commands to execute. Since this mapping is part of the test script, which is part of the U-Boot code/branch, this approach is more scalable. It also feels cleaner, since again boardenv is only providing data, rather than test logic. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: dfu: error out if USB device already existsStephen Warren2016-01-28-0/+6
| | | | | | | | | | | | | | | | | | The DFU test requests U-Boot configure its USB controller in device mode, then waits for the host machine to enumerate the USB device and create a device node for it. However, this wait can be fooled if the USB device node already exists before the test starts, e.g. if some previous software stack already configured the USB controller into device mode and never de-configured it. This "previous software stack" could even be another test/py test, if U-Boot's own USB teardown does not operate correctly. If this happens, dfu-util may be run before U-Boot is ready to serve DFU commands, which may cause false test failures. Enhance the dfu test to fail if the device node exists before it is expected to. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: add a networking testStephen Warren2016-01-28-0/+153
| | | | | | | | | | | This tests: - dhcp (if indicated by boardenv file). - Static IP network setup (if provided by boardenv file). - Ping. - TFTP get. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move find_ram_base() into u_boot_utilsStephen Warren2016-01-28-2/+3
| | | | | | | | find_ram_base() is a shared utility function, not a core part of the U-Boot console interaction. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add DFU testStephen Warren2016-01-28-0/+262
| | | | | | | | | | | | | | | Add a test of DFU functionality to the Python test suite. The test starts DFU in U-Boot, waits for USB device enumeration on the host, executes dfu-util multiple times to test various transfer sizes, many of which trigger USB driver edge cases, and finally aborts the DFU command in U-Boot. This test mirrors the functionality previously available via the shell scripts in test/dfu, and hence those are removed too. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: ums: add filesystem-based testingStephen Warren2016-01-28-48/+188
| | | | | | | | | | | | | | | | | | | | | Enhance the UMS test to optionally mount a partition and read/write a file to it, validating that the content written and read back are identical. This enhancement is backwards-compatible; old boardenv contents that don't define the new configuration data will cause the test code to perform as before. test/ums/ is deleted since the Python test now performs the same testing that it did. The code is also re-written to make use of the recently added utility module, and split it up into nested functions so the overall logic of the test process can be followed more easily without the details cluttering the code. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move U-Boot respawn trigger to the test coreStephen Warren2016-01-28-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, U-Boot was lazilly (re-)spawned if/when a test attempted to interact with it, and no active connection existed. This approach was simple, yet had the disadvantage that U-Boot might be spawned in the middle of a test function, e.g. after the test had already performed actions such as creating data files, etc. In that case, this could cause the log to contain the sequence (1) some test logs, (2) U-Boot's boot process, (3) the rest of that test's logs. This isn't optimally readable. This issue will affect the upcoming DFU and enhanced UMS tests. This change converts u_boot_console to be a function-scoped fixture, so that pytest attempts to re-create the object for each test invocation. This allows the fixture factory function to ensure that U-Boot is spawned prior to every test. In practice, the same object is returned each time so there is essentially no additional overhead due to this change. This allows us to remove: - The explicit ensure_spawned() call from test_sleep, since the core now ensures that the spawn happens before the test code is executed. - The laxy calls to ensure_spawned() in the u_boot_console_* implementations. The one downside is that test_env's "state_ttest_env" fixture must be converted to a function-scoped fixture too, since a module-scoped fixture cannot use a function-scoped fixture. To avoid overhead, we use the same trick of returning the same object each time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add a test for the sleep commandStephen Warren2016-01-20-0/+24
| | | | | | | | Execute "sleep", and validate that it sleeps for approximately the correct amount of time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the ums commandStephen Warren2016-01-20-0/+94
| | | | | | | | | This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the shell if commandStephen Warren2016-01-20-0/+154
| | | | | | | | | | | | | | Migrate all most tests from command_ut.c into the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code from command_ut. Some tests in command_ut.c can only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add test of basic shell functionalityStephen Warren2016-01-20-0/+42
| | | | | | | | | | | This tests whether the following features of the U-Boot shell: - Execution of a directly entered command. - Compound commands (; delimiter). - Quoting of arguments containing spaces. - Executing commands from environment variables. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the md/mw commandsStephen Warren2016-01-20-0/+36
| | | | | | | | | | | | | This tests whether md/mw work, and affect each-other. Command repeat is also tested. test/cmd_repeat.sh is removed, since the new Python-based test does everything it used to. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add test of setenv/printenv/echoStephen Warren2016-01-20-0/+221
| | | | | | | | This tests basic environment variable functionality. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test that sandbox exits when askedStephen Warren2016-01-20-0/+24
| | | | | | | | | Test the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Implement pytest infrastructureStephen Warren2016-01-20-0/+43
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are: - Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect. - There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C. - It is reasonably simple to interact with U-Boot in this way. A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too. The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks. See README.md for more details! Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Tested-by: Michal Simek <michal.simek@xilinx.com> Tested-by: Simon Glass <sjg@chromium.org> Acked-by: Simon Glass <sjg@chromium.org> #v3
generated by cgit v1.1 at 2024-11-08 16:14:27 +0000