Commit Graph

485 Commits

Author SHA1 Message Date
Marti Bolivar 3a24890a24 testing: add shippable integration
This ought to run the tests across a good enough spread of supported
Python versions to get started, and collect coverage statistics.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-21 14:54:29 -05:00
Marti Bolivar ce3f4fd565 tests_requirements.txt: add pytest-cov
This enables gathering of code coverage data during testing.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-21 14:54:29 -05:00
Marti Bolivar 04accde952 tests: add nrfjprog runner tests
Add an exhaustive test suite.

Check expected results for all combinations of runner constructor
parameters, both when instantiating the runner directly via its
constructor and when using command line arguments via its create()
method. Ensure consistent results.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Marti Bolivar f86da451f9 tests: add pyocd tests
Add test coverage for flash, debug, and debugserver commands, using
runners created directly with their constructors, as well as via
command line arguments, with methods that would run syscalls mocked
out.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Marti Bolivar 9b7d58343f Add py.test-based test infrastructure
Follow along with what the cool kids are doing and add py.test
integration.

This patch doesn't add any test cases, but sets up packages to
test the runner classes as additional work.

To run the test suite, use:

$ python3 setup.py test

Since there are no tests, these pass.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Marti Bolivar 304776695b Move install_requires contents to requirements.txt
This makes it easier to do editable installs or work on West in a
virtualenv.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Marti Bolivar 1e809d5757 Move sources into a src/ directory
As prep work for adding test cases, follow pytest best practices by
having a src/ directory:

https://docs.pytest.org/en/latest/goodpractices.html#tests-outside-application-code

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Gautier Seidel 1f35506a5e Apply Zephyr commit 1952c56e7
NOTE: This applies Zephyr commit 1952c56e7 ("esp32: add abitily to
      flash bootloader"), which was merged without going through this
      tree first.

'make flash' also flashes the bootloader

Signed-off-by: Gautier Seidel <gautier.seidel@tado.com>
Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Wayne Ren d0f184412a Apply Zephyr commit 4a8393dd6
NOTE: This applies the West-specific portions of Zephyr commit
      4a8393dd6 ("esp32: add abitily to flash bootloader"), which was
      merged without going through this tree first.

add nsim runner in west

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Jamie McCrae 112b87759e Apply Zephyr commit 60e97de58
NOTE: This applies Zephyr commit 60e97de58 ("scripts: runner:
      nrfjprog: Allow specifying serial number of nrfjprog"), which
      was merged without going through this tree first.

This allows for scripts using nrfjrog to specify the serial number of
the attached device to use instead of showing a list of available
devices by adding an optional parameter.

Signed-off-by: Jamie McCrae <jamie.mccrae@lairdtech.com>
Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-08-20 09:17:57 -07:00
Marti Bolivar 8e22ca68c3 Add bootstrap script.
Add a separate package for bootstrapping West and a Zephyr manifest
repository, and have the setuptools integration install that instead
of the "real" West.

Signed-off-by: Marti Bolivar <marti.f.bolivar@gmail.com>
2018-06-11 21:43:50 -04:00
Marti Bolivar 4d75a749d0 .gitignore: add __pycache__/
Clean up the status output.

Signed-off-by: Marti Bolivar <marti.f.bolivar@gmail.com>
2018-06-11 21:43:50 -04:00
Marti Bolivar 321ab2e179 runner: core: fix docstrings for ReST integration
A couple of docstrings are causing issues when imported into Sphinx;
fix these up.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-08 11:52:03 -04:00
Marti Bolivar 2e9e6b255d Add setuptools integration.
Follow along with the tutorial:

https://packaging.python.org/tutorials/packaging-projects/

The version number is because of repeated uploads to TestPyPI.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-06 12:21:14 -04:00
Marti Bolivar d8542f297c main: tweaks to enable setuptools integration
The setuptools entry_points kwarg can generate platform-specific
scripts to run. These call the entry points with no arguments,
however, so tweak our main() to use sys.argv and its caller to stop
passing it.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-06 12:21:14 -04:00
Marti Bolivar 2570866cf2 Move everything into a west/ directory
This is necessary to properly configure this as a Python package with
setuptools integration.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-06 12:21:14 -04:00
Marti Bolivar d656eb4a32 runner: add --erase flag to nrfjprog.py
By default, use a sector erase to flash boards with nrfjprog.py.

To allow getting the old behavior, add an --erase flag that works the
exact same way as the corresponding jlink.py argument: if present, a
full flash erase is done before programming. If absent, only a sector
erase is done.

Fixes zephyrproject-rtos/zephyr#6147

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-06-04 08:42:55 -04:00
Marti Bolivar 894aedbc0e runner: nios2.py: fix create() and constructor
The constructor call is missing a mandatory argument. The constructor
itself has a typo.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-04 08:41:34 -04:00
Marti Bolivar 323b96a917 cmake.py: ensure empty strings are empty lists
Ensure that CmakeCache.get_list() returns an empty list when the cache
entry is the empty string.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-06-04 08:41:34 -04:00
Marti Bolivar a65b8b87f2 main: downgrade missing ZEPHYR_BASE to a warning
This is not strictly necessary for flashing and debugging, and is
causing issues in Linux environments where users run "make flash" as
root instead of installing udev rules.

Fixes: #7676

(Though we will need to revisit this when adding commands that run
CMake).

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:36 -04:00
Marti Bolivar 53fc351eb1 cmd: add context-sensitive runner help
Though commands like "west flash -h" now have help for generic runner
configuration options, runner-specific context is not printed.

In order to print this information, we would ideally want to know the
currently available runners from a build directory. Otherwise, we
can't print the current cached configuration, and the user will likely
be overwhelmed by a giant list of options etc. available for all the
runners in the package.

However, we can't print that information out without re-doing the
build, which is not safe to do when the user just gives '--help'.

To provide more complete help without causing side effects in the
default help output, add a new -H/--context option, which explicitly
re-runs the build (unless --skip-rebuild was given), parses the cache,
and prints context-sensitive help. This can be combined with the -r
option to restrict help to a particular runner.

Examples:

- Print context for all available flash runners:

      west flash -H --build-dir build-frdm_k64f/

- Print context for just one runner:

      west flash -H --build-dir build-frdm_k64f/ -r jlink

- Print context for all available debug runners, if current
  working directory is a build directory:

      west debug -H

If no context is available because there is no CMake cache file, this
command can still be used to obtain generic information about
runners. It emits a warning in this case.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:36 -04:00
Marti Bolivar 05358c9789 runner: add build directory to runner configuration
Add the build directory to the central runner configuration.
This is commonly useful information.

The first place we can use it is to eliminate guessing the current
working directory when building objects to parse .config.

It's not necessary to modify the build system for this, so leave the
relevant command line flag among the general options.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:36 -04:00
Marti Bolivar dfd1bf48bc runner: introduce common runner configuration
Continue better integration of the runner subpackage into west by
moving the common runner configuration options into the command
core. This allows commands like "west flash -h" to display help for
common overrides like --kernel-hex.

Adjustments needed to make this happen are:

- Change the build system to separate common configuration values from
  runner-specific options and arguments

- Prepare the runner core by defining a new RunnerConfig class that
  represents the common configuration, and accepting that from a new
  create() method, which replaces create_from_args().

- Convert all concrete runner classes to use the new style of
  argument parsing and initialization.

- Group the command options appropriately for help output readability

There's still a bit of tool-specific stuff in the common
configuration (gdb and openocd configuration in particular); a more
generic way to deal with that will be necessary to better support
things like non-GDB debuggers, but that's out of scope of this patch.

All the runner-specific options are still in the runner packge, which
currently prevents them from being included in "west flash -h" etc.
Fixing that is also out of scope of this patch.

This has the ancillary benefit of getting rid of the legacy 'debug'
argument to ZephyrBinaryRunner, which is no longer appropriate since
verbose debug logging is handled by log.py in west.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:36 -04:00
Marti Bolivar 54e48e8bf5 runner: convert runner to use log module
Replace all informational messages with calls to log functions.

Cases which must interact via the terminal and standard output are not
modified.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar 98dc26cfd2 runner: fix some intel_s1000 runner issues
There are a few issues with the intel_s1000 runner:

- it doesn't attach to the client when running debug (this is
  a part of the command's contract specified in core.py)

- it uses hard-coded sleeps when running subprocesses instead of
  waiting for them to terminate

- it re-implements pre-existing popen_ignore_int() functionality (in a
  way that is not portable to Windows) rather than using it directly

Fix these issues.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar 3bbcb7ab4d runner: clean up intel_s1000 runner
Make the flake8 linter happy, spell the board name with the correct
case in the module docstring, and keep the usual camel case naming
convention used for classes.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar 073ce08135 runner: add subprocess.call wrapper to runner classes
There's a use case for it in intel_s1000.py

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar e7bd3178c4 runner: trivial fix to runner comments
Trivial fix to runner core.py.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar bf7026debd runner: remove command from runner arguments
This is redundant information.  The command is already known from the
top-level west command line. We can just feed it to run without
inserting it on the command line as well, which is safe to do now that
zephyr_flash_debug.py is gone.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar b1477f67d6 cmd: add flash, debug, debugserver commands
When run without any arguments, the commands work the same way that
their CMake equivalents do. For example, if using the Ninja CMake
generator, these are equivalent:

    west flash        <----> ninja flash
    west debug        <----> ninja debug
    west debugserver  <----> ninja debugserver

Like CMake's build tool mode, you can also run them from any directory
in the system by passing the path to the build directory using
--build-dir (-d):

    west flash -d build/my-board

The commands will run the CMake-generated build system, so they keep
dependencies up to date and users don't have to manually compile
binaries between running CMake and using this tool.

The commands also support important use cases that CMake can't:

1) Any arguments not handled by 'west flash' et al. are passed to the
   underlying runner. For example, if the runner supports --gdb-port,
   the default can be overridden like so:

   west debugserver --gdb-port=1234

   Command processing by the 'west' command can also be halted using
   '--'; anything after that point (even if it's an option recognized
   by the west command) will be passed to the runner. Example:

   west debug -- --this-option-goes-to-the-debug-runner=foo

2) Any runner supported by the board can be selected at runtime using
   the -r (--runner) option. For example, if the board's flash runner
   defaults to nrfjprog but jlink is supported as well, it can be
   selected with:

   west flash -r jlink

3) The runner configuration can be persisted elsewhere, edited
   offline, and selected at runtime, using --cmake-cache (-c):

   west flash -c /home/me/some/other/CMakeCache.txt

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar d514bb2db8 runner: add get_runner_cls()
Add a utility method for getting a runner class given its name.  This
will be used in an upcoming patch which adds a command for printing
runner information.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar 7e510aff96 Remove redundant quote_sh_list definition
There is a copy in the west util module; now that runner is a
subpackage of west, just import it from there.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar 0088ad57ab Make runner a west subpackage
This is a stepping-stone to adding runner functionality into west
itself.

Since all of the runner tools assume a Zephyr build directory layout,
this doesn't put anything generic into a Zephyr-specific tool.

Make minimal adjustments to zephyr_flash_debug.py to keep existing
build system targets working unmodified.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar a688118800 Add cmake utility module
This will be used in subsequent patches when adding commands that
interface with the runner package. These need to do things like
ensuring the CMake build is up to date, parsing the cache, etc.

To keep that interface clean, provide this functionality in a separate
module.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:35 -04:00
Marti Bolivar c662ddf8a6 Create meta-tool package, "west"
We have agreed to develop a meta-tool named "west", which will be a
swiss-army knife of Zephyr development. It will support use cases like
building, flashing and debugging; bootloader integration; emulator
support; and integration with multiple git repositories.

The basic usage for the tool is similar to git(1):

    west [common opts] <command-name> [command opts] [<command args>]

There are common options, such as verbosity control, followed by a
mandatory sub-command. The sub-command then takes its own options and
arguments.

This patch adds the basic framework for this tool, as follows:

- a Python 3 package named 'west'. There is no PyPI
  integration for now; the tool will be improving quickly, so we need
  to keep users up to date by having it in tree.
- an main entry point, main.py, and a package-level shim, __main__.py
- a cmd subpackage, which defines the abstract base class for commands
- logging (log.py)
- catch-all utilities (util.py)

Windows and Unix launchers so users can type "west" to run the tool
after sourcing the appropriate zephyr-env script for their
environment are in the Zephyr tree.

Subsequent patches will start to add individual commands.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2018-05-23 10:27:10 -04:00