We have been dropping lines after finding a fault which resulted in
missing information in the log. Make sure we continue and only report
failure at the end of the execution.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not close console after PASS is reported, wait a bit for any
remaining messages from the tests, sometimes we have faults that need to
be parsed.
This now works for Qemu handler, support for other handlers to follow.
Fixes#9646
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Since a distributions name for the ply package may change, the only
constant name which can be used for a recommendation is the name on
PyPi which happens to be "ply".
Signed-off-by: Daniel Egger <daniel@eggers-club.de>
Add new kwyboard to board definition to allow blacklisting boards. This
is needed when a board is broken causing CI to fail without a fix in
sight.
Add:
sanitycheck: false
to the board yaml file to disable the board. By default, the value is
set to true.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fail in tests where we have an OOPS or a panic. Right now and in many
cases we continue and test case might be reported as PASS.
Cases that have the tag ignore_faults will ignore those faults (cases
that are testing faults for example).
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- Some tests start with test_, some do not, so make sure we parse both.
- Parse skipped tests
- Improve handling of test case identifier
- Handle Exceptions in device handler
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Parse the test results and create a test report with more granular
results that can be imported to into test management/reporting system.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Allow for any toolchain variants to be useable with sanitycheck without
the need for them to be registered.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add 2 classes, one to handle the current TestCase scenario, and one more
for handling generic Console with regex matching.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some boards are supported natively by qemu. This option will allow us to
run tests using those platforms directly without having to go via a
dedicated qemu board definition.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This keyword would mean that a special harness is needed to run the
tests sucessfully. This can be as simple as a loopback wiring or a
complete hardware test setup for sensor and IO testing. It is free form
initially and would be changed to be an enum once we have more values in
place.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
when running multiple instances of sanitycheck, allow placing the
parsetab.py in a customer location that can be set using an environment
variable.
export PARSETAB_DIR=/tmp/
run sanitycheck and the parsetab.py will be placed in /tmp/.
Fixes#4513
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Simplify parsing of yaml structures and remove usage of cp which was for
the ConfigParser used for ini files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Support new keywords in testcase.yaml that would allow us to inject
configuration options to be merged with default configuration instead of
having to provide a prj.conf for each variant of the test which is very
difficult to keep in sync. Sanitycheck script will create an overlay
file that is merged during the build process.
This is now done using the extra_configs option which is a yaml list of
option with the values, for example:
extra_configs:
- CONFIG_XXXX=y
- CONFIG_YYYY=y
With this option we can have multiple tests that for example run on
hardware with different values. This type of testing is good on HW but
it does not make sense to be built in normal sanitycheck operation
because it will be just rebuilding the same code with different values.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit changes the syntax of the testcase files and changes the
behaviour and configuration of the sanitycheck script.
To avoid having multiple files with different syntax for boards,
samples, tests; this change unifies the syntax and uses YAML instead of
INI.
We maintain the current keywords used in the old syntax and maintain the
flexibility of adding tests with different configuration by using YAML
list configuration. On top of that, the following features are added:
- We now scan for board configurations in the boards directory and look
for a YAML file describing a board and how it should be tested. This
eliminates the need for listing boards per architecture in a special ini
file under scripts/.
- We define hardware information charachterstics in the board YAML file
that helps identifying if a certain test should run on that board or
not. For example, we can specify the available RAM in the board and
filter tests that would require more RAM than the board can handle.
- Boards can be set as default for testing meaning that we always run a
test case (build and run of possible) when sanitycheck is called without
any arguments. Previously this was done only by selecting the first
board defined for a specific architecture.
- Tests can be configured to run on all possible boards, this is to make
sure we always build some basic tests for all boards to catch issues
with the core kernel features.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- board name olimex_stm32_e407
- CPU STM32F407ZGT6 Cortex M4
- LED/BUTTON support
- Console on USART1 with 8n1 115200 baud
Signed-off-by: Erwin Rol <erwin@erwinrol.com>
Add necessary board files, pinmux and device tree in order to have a
usable debug console.
Origin: Original
Change-Id: I43a9d278c3f2c936a714263626722f630367b663
Signed-off-by: Florian Vaussard <florian.vaussard@heig-vd.ch>
The stm32f4discovery was incorrectly named and should have been
stm32f4_disco. Also added 96b_carbon_nrf51 that was missing from the
list.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add configuration, pinmux, dts and documentation for the STM32L496G
Discovery board based on the STM32L496AG SoC.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Now that we can specify what toolchain is intended for each
SOC, enable some more SOCs to be built.
A full sanitycheck run will require the installation of both
RF-2016.4 and RG-2016.4 releases.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add necessary board files, pinmux and device tree in order to have a
usable debug console.
Origin: Original
Change-Id: I280320700352fd36a544c03f4e57d2eeec2449e5
Signed-off-by: Florian Vaussard <florian.vaussard@heig-vd.ch>
This core configuration was removed from the tree since it cannot
implement irq_offload().
Remove an orphaned block in xtesna.ini.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add configuration, dts and documentation for the Nucleo L432KC board
based on the STM32L432KC SoC.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
This commit provides support for disco_l475_iot1 board
Pinmux driver is provided with initial support definitions
Change-Id: I17b637a8ba0b033014969eca8fffe76319c47c52
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
In order to allow the use of such board, a very preliminar port was
developed. It consists of board files, as well as pinmux, uart, gpio,
spi drivers and device tree files.
Change-Id: I5753064e39e0b023cf4481744c176de26d8dbebb
Signed-off-by: Gustavo Denardin <gustavo.denardin@gmail.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
CC3220SF_LAUNCHXL effectively replaces the CC3200_LAUNCHXL,
with support for the CC3220SF SoC, which is an update for
the CC3200 SoC.
This is supported by the Texas Instruments CC3220 SDK.
Jira: ZEP-1958
Change-Id: I2484d3ee87b7f909c783597d95128f2b45db36f2
Signed-off-by: Gil Pitney <gil.pitney@linaro.org>
Anas Nashif