summaryrefslogtreecommitdiff
path: root/test
Commit message (Collapse)AuthorAgeLines
...
* video: Use fractional units for X coordinatesSimon Glass2016-01-30-2/+2
| | | | | | | | | | | | | | With anti-aliased fonts we need a more fine-grained horizontal position than a single pixel. Characters can be positioned to start part-way through a pixel, with anti-aliasing (greyscale edges) taking care of the visual effect. To cope with this, use fractional units (1/256 pixel) for horizontal positions in the text console. Signed-off-by: Simon Glass <sjg@chromium.org> [agust: rebased] Signed-off-by: Anatolij Gustschin <agust@denx.de>
* test/py: dfu: allow boardenv to specify test sizesStephen Warren2016-01-28-2/+10
| | | | | | | | | | | | | Allow the env__dfu_configs boardenv data to specify the set of DFU transfer sizes to test. Manually specifying test sizes is useful if you wish to test multiple DFU configurations (e.g. SD card ext4 filesystem, SD card whole raw partition, RAM, etc.), but don't want to test every single transfer size on each, to avoid bloating the overall time taken by testing. If the boardenv doesn't specify a set of sizes, the built-in list is used as a default, preserving backwards-compatibility. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix a couple typos in commentsStephen Warren2016-01-28-2/+2
| | | | | | | s/updata/update/. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: run sandbox in source directoryStephen Warren2016-01-28-3/+7
| | | | | | | | Some unit tests expect the cwd of the sandbox process to be the root of the source tree. Ensure that requirement is met. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: pass test DTB to sandboxStephen Warren2016-01-28-1/+6
| | | | | | | This is required for at least "ut dm" to operate correctly. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: correctly log xfail/xpass testsStephen Warren2016-01-28-24/+74
| | | | | | | | | | Tests can complete in passed, skipped, xpass, xfailed, or failed, states. Currently the U-Boot log generation code doesn't handle the xfailed or xpass states since they aren't used. Add support for the remaining states. Without this, tests that xfail end up being reported as skipped. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: detect another "bad pattern" in console outputStephen Warren2016-01-28-0/+2
| | | | | | | | | | | | Many error situations in U-Boot print the message: ### ERROR ### Please RESET the board ### Add this to the list of bad patterns the test system detects. One practical advantage of this change is to detect the case where sandbox is told to use a particular DTB file, and the file cannot be opened. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: check for bad patterns everywhere we waitStephen Warren2016-01-28-7/+19
| | | | | | | | | | | | | | | | Currently, bad patterns are only honored when executing a shell command. Other cases, such as the initial boot-up of U-Boot or when interacting with command output rather than gathering all output prior to the shell prompt, do not currently look for bad patterns in console output. This patch makes sure that bad patterns are honored everywhere. One benefit of this change is that if U-Boot sandbox fails to start up, the error message it emits can be caught immediately, rather than relying on a (long) timeout when waiting for the expected signon message and/or command prompt. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test.py: calculate bad patterns on change onlyStephen Warren2016-01-28-24/+25
| | | | | | | | | | A future patch will use the bad_patterns array in multiple places. Rather than duplicating the code to calculate it, or even sharing it in a function and simply calling it redundantly when nothing has changed, only re-calculate the list when some change is made to it. This reduces work. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix spawn.expect multiple match handlingStephen Warren2016-01-28-1/+1
| | | | | | | | | | | Multiple patterns may be passed to spawn.expect(). The pattern which matches at the earliest position should be designated as the match. This aspect works correctly. When multiple patterns match at the same position, priority should be given the the earliest entry in the list of patterns. This aspect does not work correctly. This patch fixes it. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/dm: clear unit test failure count each runStephen Warren2016-01-28-0/+2
| | | | | | | | | | The ut command prints a test failure count each time it is executed. This is stored in a global variable which is never reset. Consequently, the printed failure count accumulates across runs. Fix this by clearing the counter each time "ut" is invoked. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Provide custom IDs when parametrizing testsStephen Warren2016-01-28-1/+13
| | | | | | | | | | | | | | | | | | | | | When pytest generates the name for parametrized tests, simple parameter values (ints, strings) get used directly, but more complex values such as dicts are not handled. This yields test names such as: dfu[env__usb_dev_port0-env__dfu_config0] dfu[env__usb_dev_port0-env__dfu_config1] Add some code to extract a custom fixture ID from the fixture values, so that we end up with meaningful names such as: dfu[micro_b-emmc] dfu[devport2-ram] If the boardenv file doesn't define custom names, the code falls back to the old algorithm. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: Quote consistencyStephen Warren2016-01-28-42/+42
| | | | | | | | | | | When converting test/py from " to ', I missed a few places (or added a few inconsistencies later). Fix these. Note that only quotes in code are converted; double-quotes in comments and HTML are left as-is, since English and HTML use " not '. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: use " for docstringsStephen Warren2016-01-28-245/+245
| | | | | | | | | | | Python's coding style docs indicate to use " not ' for docstrings. test/py has other violations of the coding style docs, since the docs specify a stranger style than I would expect, but nobody has complained about those yet:-) Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: make net test aware of USB and PCI enumerationStephen Warren2016-01-28-13/+15
| | | | | | | | | | | | | | | | | | | | | | The existing net test executes a list of commands supplied by boardenv variable env__net_pre_commands. The idea was that boardenv would know whether the Ethernet device was attached to USB, PCI, ... and hence was the best place to put any commands required to probe the device. However, this approach doesn't scale well when attempting to use a single boardenv across multiple branches of U-Boot, some of which require "pci enum" to enumerate PCI and others of which don't, or don't /yet/ simply because various upstream changes haven't been merged down. This patch updates the test to require that the boardenv state which HW features are required for Ethernet to work, and lets the test itself map that knowledge to the set of commands to execute. Since this mapping is part of the test script, which is part of the U-Boot code/branch, this approach is more scalable. It also feels cleaner, since again boardenv is only providing data, rather than test logic. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: dfu: error out if USB device already existsStephen Warren2016-01-28-0/+6
| | | | | | | | | | | | | | | | | | The DFU test requests U-Boot configure its USB controller in device mode, then waits for the host machine to enumerate the USB device and create a device node for it. However, this wait can be fooled if the USB device node already exists before the test starts, e.g. if some previous software stack already configured the USB controller into device mode and never de-configured it. This "previous software stack" could even be another test/py test, if U-Boot's own USB teardown does not operate correctly. If this happens, dfu-util may be run before U-Boot is ready to serve DFU commands, which may cause false test failures. Enhance the dfu test to fail if the device node exists before it is expected to. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* test/py: make crash detection more robustStephen Warren2016-01-28-13/+7
| | | | | | | | | | | | | | | | | | | | test/py contains logic to detect the target crashing and rebooting by searching the console output for a U-Boot signon message, which will presumably be emitted when the system boots after the crash/reset. Currently, this logic only searches for the exact signon message that was printed by the U-Boot version under test, upon the assumption that binary is written into flash, and hence will be the version booted after any reset. However, this is not a valid assumption; some test setups download the U-Boot-under-test into RAM and boot it from there, and in such a scenario an arbitrary U-Boot version may be located in flash and hence run after any reset. Fix the reset detection logic to match any U-Boot signon message. This prevents false negatives. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add a networking testStephen Warren2016-01-28-0/+153
| | | | | | | | | | | This tests: - dhcp (if indicated by boardenv file). - Static IP network setup (if provided by boardenv file). - Ping. - TFTP get. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move find_ram_base() into u_boot_utilsStephen Warren2016-01-28-39/+41
| | | | | | | | find_ram_base() is a shared utility function, not a core part of the U-Boot console interaction. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add DFU testStephen Warren2016-01-28-197/+262
| | | | | | | | | | | | | | | Add a test of DFU functionality to the Python test suite. The test starts DFU in U-Boot, waits for USB device enumeration on the host, executes dfu-util multiple times to test various transfer sizes, many of which trigger USB driver edge cases, and finally aborts the DFU command in U-Boot. This test mirrors the functionality previously available via the shell scripts in test/dfu, and hence those are removed too. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: ums: add filesystem-based testingStephen Warren2016-01-28-261/+188
| | | | | | | | | | | | | | | | | | | | | Enhance the UMS test to optionally mount a partition and read/write a file to it, validating that the content written and read back are identical. This enhancement is backwards-compatible; old boardenv contents that don't define the new configuration data will cause the test code to perform as before. test/ums/ is deleted since the Python test now performs the same testing that it did. The code is also re-written to make use of the recently added utility module, and split it up into nested functions so the overall logic of the test process can be followed more easily without the details cluttering the code. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add various utility codeStephen Warren2016-01-28-0/+190
| | | | | | | | Add various common utility functions. These will be used by a forthcoming re-written UMS test, and a brand-new DFU test. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: optionally ignore errors from shell commandsStephen Warren2016-01-28-2/+6
| | | | | | | | | | Sometimes it's useful to run shell commands and ignore any errors. One example might be cleanup logic; if a test-case experiences an error, the cleanup logic might experience an error too, and we don't want that error to mask the original error, so we want to ignore the subsequent error. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: log when tests send CTRL-CStephen Warren2016-01-28-0/+1
| | | | | | | | | | Write a note to the log file when a test sends CTRL-C to U-Boot. This makes it easier to follow what's happening in the logs, especially since U-Boot doesn't echo the character back to its output, so there's no other signal of what's going on. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: drain console log at the end of any failed testStephen Warren2016-01-28-0/+39
| | | | | | | | | | | | | | | | Tests may fail for a number of reasons, and in particular for reasons other than a timeout waiting for U-Boot to print expected data. If the last operation that a failed test performs is not waiting for U-Boot to print something, then any trailing output from U-Boot during that test's operation will not be logged as part of that test, but rather either along with the next test, or even thrown away, potentiall hiding clues re: the test failure reason. Solve this by explicitly draining (and hence logging) the U-Boot output in the case of failed tests. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: move U-Boot respawn trigger to the test coreStephen Warren2016-01-28-12/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change, U-Boot was lazilly (re-)spawned if/when a test attempted to interact with it, and no active connection existed. This approach was simple, yet had the disadvantage that U-Boot might be spawned in the middle of a test function, e.g. after the test had already performed actions such as creating data files, etc. In that case, this could cause the log to contain the sequence (1) some test logs, (2) U-Boot's boot process, (3) the rest of that test's logs. This isn't optimally readable. This issue will affect the upcoming DFU and enhanced UMS tests. This change converts u_boot_console to be a function-scoped fixture, so that pytest attempts to re-create the object for each test invocation. This allows the fixture factory function to ensure that U-Boot is spawned prior to every test. In practice, the same object is returned each time so there is essentially no additional overhead due to this change. This allows us to remove: - The explicit ensure_spawned() call from test_sleep, since the core now ensures that the spawn happens before the test code is executed. - The laxy calls to ensure_spawned() in the u_boot_console_* implementations. The one downside is that test_env's "state_ttest_env" fixture must be converted to a function-scoped fixture too, since a module-scoped fixture cannot use a function-scoped fixture. To avoid overhead, we use the same trick of returning the same object each time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: fix timeout to be absoluteStephen Warren2016-01-28-1/+6
| | | | | | | | | | | | | | Currently, Spawn.expect() imposes its timeout solely upon receipt of new data, not on its overall operation. In theory, this could cause the timeout not to fire if U-Boot continually generated output that did not match the expected patterns. Fix the code to additionally impose a timeout on overall operation, which is the intended mode of operation. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Lukasz Majewski <l.majewski@samsung.com> Acked-by: Simon Glass <sjg@chromium.org>
* dm: syscon: Allow finding devices by driver dataSimon Glass2016-01-24-0/+17
| | | | | | | | We have a way to find a regmap by its syscon driver data value. Add the same for syscon itself. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
* dm: video: test: Test that bitmap display works correctlySimon Glass2016-01-20-0/+54
| | | | | | | | Add a test for the 'bmp' command. Test both the uncompressed and compressed versions of the file, since they use different code paths. Signed-off-by: Simon Glass <sjg@chromium.org> Acked-by: Anatolij Gustschin <agust@denx.de>
* dm: video: test: Add tests for rotated consolesSimon Glass2016-01-20-0/+27
| | | | | | | Test that text is displayed correctly when the console is rotated. Signed-off-by: Simon Glass <sjg@chromium.org> Acked-by: Anatolij Gustschin <agust@denx.de>
* dm: video: test: Add tests for the video uclassSimon Glass2016-01-20-0/+191
| | | | | | | | | Add tests that check that the video console is working correcty. Also check that text output produces the expected result. Test coverage includes character output, wrapping and scrolling. Signed-off-by: Simon Glass <sjg@chromium.org> Acked-by: Anatolij Gustschin <agust@denx.de>
* test/py: add a test for the sleep commandStephen Warren2016-01-20-0/+24
| | | | | | | | Execute "sleep", and validate that it sleeps for approximately the correct amount of time. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the ums commandStephen Warren2016-01-20-0/+94
| | | | | | | | | This test invokes the "ums" command in U-Boot, and validates that a USB storage device is enumerated on the test host system, and can be read from. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the shell if commandStephen Warren2016-01-20-116/+154
| | | | | | | | | | | | | | Migrate all most tests from command_ut.c into the Python test system. This allows the tests to be run against any U-Boot binary that supports the if command (i.e. where hush is enabled) without requiring that binary to be permanently bloated with the code from command_ut. Some tests in command_ut.c can only be executed from C code, since they test internal (more unit-level) features of various U-Boot APIs. The migrated tests can all operate directly from the U-Boot console. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add test of basic shell functionalityStephen Warren2016-01-20-16/+42
| | | | | | | | | | | This tests whether the following features of the U-Boot shell: - Execution of a directly entered command. - Compound commands (; delimiter). - Quoting of arguments containing spaces. - Executing commands from environment variables. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test the md/mw commandsStephen Warren2016-01-20-29/+36
| | | | | | | | | | | | | This tests whether md/mw work, and affect each-other. Command repeat is also tested. test/cmd_repeat.sh is removed, since the new Python-based test does everything it used to. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: add test of setenv/printenv/echoStephen Warren2016-01-20-4/+221
| | | | | | | | This tests basic environment variable functionality. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: test that sandbox exits when askedStephen Warren2016-01-20-0/+24
| | | | | | | | | Test the sandbox port's implementation of the reset command and SIGHUP handling. These should both cause the U-Boot process to exit gracefully. Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
* test/py: Implement pytest infrastructureStephen Warren2016-01-20-0/+2090
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are: - Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect. - There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C. - It is reasonably simple to interact with U-Boot in this way. A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too. The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks. See README.md for more details! Signed-off-by: Stephen Warren <swarren@wwwdotorg.org> Signed-off-by: Stephen Warren <swarren@nvidia.com> Tested-by: Michal Simek <michal.simek@xilinx.com> Tested-by: Simon Glass <sjg@chromium.org> Acked-by: Simon Glass <sjg@chromium.org> #v3
* test/fs: error case fixes/enhancementsStephen Warren2016-01-13-1/+13
| | | | | | | | | | | | - Use "mkdir -p" to avoid errors when intermediate directories are missing. - Fall back to "dd" when "fallocate" fails. For example, fallocate isn't supported on ext4. - Add error checking for test image generation. Without this, the test simply plows on spewing all kinds of errors which are hard to immediately root-cause. Signed-off-by: Stephen Warren <swarren@nvidia.com>
* block: pass block dev not num to read/write/erase()Stephen Warren2016-01-13-1/+1
| | | | | | | | | | | This will allow the implementation to make use of data in the block_dev structure beyond the base device number. This will be useful so that eMMC block devices can encompass the HW partition ID rather than treating this out-of-band. Equally, the existence of the priv field is crying out for this patch to exist. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Tom Rini <trini@konsulko.com>
* dm: test: Convert PCI tests to use the DM PCI APISimon Glass2016-01-12-9/+6
| | | | | | Update these tests to use driver model API functions. Signed-off-by: Simon Glass <sjg@chromium.org>
* dm: test: usb: Update the USB tests so that they all passSimon Glass2016-01-11-7/+16
| | | | | | | Due to a limitation removed in an earlier patch, USB tests were not seeing all the devices. Update the tests to pass now that all devices are visible. Signed-off-by: Simon Glass <sjg@chromium.org>
* dm: Disable all SPI and SPI flash testsSimon Glass2016-01-07-2/+2
| | | | | | | | | | | This subsystem has been broken since commit: 4efad20a sf: Update status reg check in spi_flash_cmd_wait_ready There has so far been no response from the maintainer, and a release is imminent. For now, let's just disable the tests. Signed-off-by: Simon Glass <sjg@chromium.org>
* CONFIG_NEEDS_MANUAL_RELOC: Fix warnings when not setTom Rini2015-12-07-0/+1
| | | | | | | | | Now that we may compile (but not link) code calling fixup_cmdtable when this is not set, we need to always have the declaration available. We should also make sure that anyone calling the function includes <command.h> as that's where the function declaration is. Signed-off-by: Tom Rini <trini@konsulko.com>
* dm: cmd: Relocate subcommands when MANUAL_RELOCMichal Simek2015-12-07-0/+14
| | | | | | | | Subcommands contain pointers to functions which are not updated when MANUAL_RELOC is enabled. This patch fix it. Signed-off-by: Michal Simek <michal.simek@xilinx.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* dm: test: usb: sandbox: Add keyboard tests for sandboxSimon Glass2015-11-19-0/+31
| | | | | | | Add a test that verifies that USB keyboards work correctly on sandbox. This verifies some additional parts of the USB stack. Signed-off-by: Simon Glass <sjg@chromium.org>
* dm: test: usb: Add a test for device reorderingSimon Glass2015-11-19-0/+50
| | | | | | | Add tests that 'usb tree' produces the right output when a device changes order on the bus. Signed-off-by: Simon Glass <sjg@chromium.org>
* dm: test: usb: Add tests for the 'usb tree' commandSimon Glass2015-11-19-0/+156
| | | | | | | Add tests that this command produces the right output, even when a rescan results in a device disappearing from the bus. Signed-off-by: Simon Glass <sjg@chromium.org>
* test: Record and silence console in testsSimon Glass2015-11-19-0/+13
| | | | | | | | | When running sandbox tests, silence the console to avoid unwanted output. Also, record the console in case tests want to check it. The -v option can be used to enable stdout during tests. Signed-off-by: Simon Glass <sjg@chromium.org>
generated by cgit v1.1 at 2024-11-10 05:45:37 +0000