Commit Graph

334 Commits

Author SHA1 Message Date
Marti Bolivar 4fe9d4a042 configuration: add delete_config()
We currently have no way to delete options once set. This is not good
for users that want to delete things so they go back to their default
values. Add a library API and tests for this; we'll extend the config
command with options that use them next.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 4636b3def2 configuration: always use configobj
Delete the shim that tried to use ConfigParser, which was introduced
as a shim during development while we didn't have a release out that
had a dependency on configobj. We do now, and it's no longer needed.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 46ca8354bd conftest.py: add license/copyright header
This was missing.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 6cf0a0236b configuration: allow environment overrides
Add three new environment variables which can be used to override the
configuration file locations: WEST_CONFIG_SYSTEM, WEST_CONFIG_GLOBAL,
and WEST_CONFIG_LOCAL.

Use them in the tests. This also has the beneficial side effect of
making sure that "real" system settings do not interfere with the test
environment.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar b94b71bbfa tests: allow cmd() to use non-standard environment
We'll use this the next patch. It's useful for isolation, because the
environment is one of the things that does live on between tests.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 06683b791c commands: config: add --list option
This prints all existing config options and their values.
It can be combined with the --system, --global, and --local options
to just list the contents of those files.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 79e7b5b3e1 commands: config: trivial refactor
Factor out read/write helpers. This is useless now but will keep the
code clean later, when we add more things this command can do.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 52f1f48de6 commands: config: error out if value is unset
This makes sense, and matches git config's behavior too. Stay silent
by default, but log.dbg() what happened.

Add a test case, too.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar b02caa6c8f commands: config: try to improve -h output
Align the description with the west.configuration docstring.

Separate the options which set the configuration file to use into
their own named argument group in the help output. List them in the
same system, global, local order that they appear in the description.

This requires using add_argument_group() instead of
add_mutually_exclusive_group(), as the latter can't take a group
name. Preserve mutual exclusion with a custom action with equivalent
semantics. This changes part of the usage from:

   [--system | --global | --local ]

to:

   [--system] [--global] [--local]

That's a bit unfortunate, but we're about to add other options that
don't belong in this group, and I think it's worth it to have these
three show up on their own.

While we're here, don't make the help output depend on the
platform. Let's keep it uniform.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar b0834c4b4c tests: add missing config system tests
Add basic tests for the behavior of the west.configuration APIs for
ConfigFile.SYSTEM, which were missing since we couldn't patch out the
location to somewhere safe.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar a3b9cd875d configuration: create missing config files
Add a helper to create the configuration file if it doesn't already
exist, and call it from update_config().

Add test cases for the API-level changes, which we can do now that we
can patch out all the config file locations.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 6a3b51d3d1 configuration: fix XDG spec compliance and support BSDs
The XDG_CONFIG_HOME variable, if set, does not specify an additional
place to look for config options: it says they should go exactly
there. Fix that. Add support for BSD platforms while we are here.

To improve testability in future patches, move the code which gets the
location for a config file into a function that we can monkeypatch
later. This will make it easier to test system level config settings.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 3291b1a5be configuration: clean up
Clean up the docstrings a bit, tweaking text and using :param xyz: to
document each parameter. Promote the "big idea" docstring to module
level.

Prefer to use `configfile` in `read_config()` to match the kwarg used
in `update_config()`, continuing to support the old way of doing
things.

Factor out a _gather_configs() helper to keep read_config() obvious.
We'll change its contents up a bit in a subsequent patch that will
make this module easier to test.

No functional changes expected.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar a58e45a0bb tests: improve test_config.py coverage and readability
Improve the test coverage and make sure that the command line
interface and module API work together as documented.

Improve readability by adding helpers, decreasing verbosity, and
marking the mandatory setup fixture autouse=True to cut boilerplate.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar 775de0f48a tests: improve test_config.py timing and behavior
Instead of bothering to create a proper west init-based installation
when testing the configuration commands, use a fixture that just
creates what is needed, and also validates module-wide assumptions
instead of repeating them in each test case.

This improves the quality of the test by only testing what is needed,
helping to isolate these tests from unrelated errors in the manifest
handling code.

It also drops the test time from ~3.5 seconds to about ~2.5 seconds in
my Linux environment.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar d2cb5d6109 tests: test_config.py cosmetics
Trivial cleanups, no functional changes expected.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar a2ae763750 tests: fix misleading conftest.py comment
We don't have west in repos_tmpdir. This is yet another old docstring
that I forgot to update when we removed the bootstrapper.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar a56319d510 util: fix misleading docstring
Make it clearer what west_dir() returns.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar c08061cef1 main: don't fail set_zephyr_base() if we can't find a manifest
Just warn instead. This is cleaner, but will also make it easier to
test commands that don't need the manifest, like 'west config', so it
helps with separation of concerns as well.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-30 11:39:05 -06:00
Marti Bolivar b99ec42568 commands: trivial west list error handling fix
format() throws IndexError on some invalid format strings. Handle that
rather than dumping stack.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-24 11:18:13 -06:00
Marti c724ee6a62 tests: fix test_project.py on Windows
This file is giving false negatives on Windows. To fix:

- test_manifest_freeze: expect (and escape) Windows paths
- test_forall: echo may be available in cmd.exe, but single-quoted
  strings sure aren't
- test_extension_command_*: handle newlines portably

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti 624880e8ff tests: conftest: use command strings exactly on Windows
This fixes test_project.py::test_list on Windows, as otherwise quoting
and splitting the relevant paths using shlex with POSIX rules behaves
incorrectly.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti 402a10a68c tests: fix test_config.py on windows
'west config' testing is giving false negatives on Windows. To fix:

- canonicalize and fix up global config path testing
- avoid passing paths to cmd(); they don't play well with
  shlex.split() and aren't necessary to make sure the config
  command is behaving properly

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti c20abf8868 util: add canon_path()
We need a common definition for canonicalizing paths; the lack of one
is causing issues on Windows.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar 3f81eb4195 manifest: project names must be unique
It is always possible to satisfy this constraint now that project URLs
may be specified explicitly.

(This restriction will be necessary to make manifest imports work
sensibly -- we should have done this before...).

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar 5415292aa4 tests: add test cases for explicit project URLs
Make sure to update comments for existing tests with modified semantics.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar a33d92596b manifest: demote defaults to a kwarg
We don't apply the west default revision correctly if defaults is
None. Since we find it convenient to let west figure out the defaults
for us, demote it to a kwarg as well and let it be None more gracefully.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar d6de81aafb manifest: support explicit project URLs
Allow each project element to explicitly specify its URL. This avoids
forcing users to name projects according to their URLs, which can be
inconvenient (and prevents us from enforcing a rule that project names
are unique).

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar 397a731a98 manifest: demote Project remote to a kwarg
We're going to add explicit project URLs to the manifest, so the
remote should no longer be a positional.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar cd9b4fa651 main.py: don't throw away tracebacks
Add a helper for saving the current traceback to a temporary file and
use it appropriately from the exception handlers in main(), This avoid
throwing away information in case the error was nondeterministic.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar b9d6600a66 main: fix CalledProcessError handling
I'm not sure this ever worked; the args field is an empty tuple even
in Python 3.4. Use cmd and returncode attributes appropriately
instead.

Don't offer the 'for a stack trace' message here anymore: this doesn't
indicate a west error, which is what that is meant to capture.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar 1c6d0d3dcb tests: fixture setup optimizations
It takes 20 seconds on my machine to run the full set of tests, which
is slow enough that testing breaks me out of flow state.

On the suspicion that creating git repositories and using the file://
protocol when cloning (which prevents use of hardlinks) is slowing
things down, use some pytest features to avoid creating git
repositories repeatedly. Also let git use hardlinks when they are
available when cloning repositories.

On my system, this brings the average of 10 runs from 20.129 seconds
spent testing to 17.649, a 12% improvement overall. Still not ideal,
but not worth throwing away, either.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar adf7e5bd2c tests: don't test pre-unification features
Don't copy the west tree; tox already installs it for us into the new
virtualenv, and we don't run any code out of a checked out repository
anymore, so doing things related to that is unnecessary.

This also makes the tests run a little bit faster (around a 5% or more
speedup on my system).

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar 34de0ad936 tests: remove empty_config.py
It's not clear what this is doing here. It doesn't begin with "test",
so it's not being tested, and it seems to be a copy of test_config.py
with some important features needed to avoid modifying the user's
configuration.

Delete it.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar e19e7e61b5 tests: flatten the hierarchy
There's no good reason to have these tests in subdirectories. Flatten
them out. Keep the directory of invalid manifests separate to keep the
directory listing clean, though.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-20 11:06:53 +02:00
Marti Bolivar c23549fe33 README: fix link to docs
Fixes: #264
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-16 13:48:29 +02:00
Marti Bolivar cb660b6e28 _bootstrap: remove it
Get rid of the separate bootstrapper. Implement 'west init' as a
regular WestCommand which combines the bootstrapper + PostInit.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar 4b880be54e commands: allow commands which don't require an installation
This will be useful to support init as a regular WestCommand.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar bee4547c81 commands: project: import more from os.path
Save some characters.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar ca19fed267 main.py: don't look for ZEPHYR_BASE if command is init
While this has no effect right now, it will make sense in
general (init --> no topdir --> it's not clear what zephyr you mean)
and will be required for making init a regular command.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar ce9899267a configuration: support read_config() outside of an installation
That's perfectly fine: load the system and global ones.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar 7153520647 setup.py: whitespace cleanup
Trivial.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar 8466885412 version: set to 0.5.99
Start v0.6.0 development. We're about to delete the bootstrapper,
which is a big enough change to need a version bump.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti 1980f707d6 West v0.5.8
This adds a couple of bootstrapper error handling fixes.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti fd53d839e9 bootstrap: check for a missing west.main
Print a more helpful message in case .west/west/src doesn't contain an
importable west.main module. (A trash .west can remain if 'west init'
fails.)

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti bb718c5a78 bootstrap: check for git before running it
Avoid issues seen on Windows where git is installed but not on PATH.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti Bolivar 5113fecc09 West v0.5.7
This includes fixes for API documentation, a configuration option handling
issue, and an improved ZEPHYR_BASE check.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar f1ca4a1d11 main.py: move west config to its own section in the help
It's not about managing multiple repositories. It's a bit weird to put it on
its own, but that's better than putting it in the wrong section.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 619d64032a main: improve ZEPHYR_BASE mismatch check
Fixes: #239
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 20d1773439 commands: fix configuration option to disable extensions
We need to be using getboolean() to parse "true" and "false" strings into their
equivalent booleans. This isn't happening, so setting the option to any
nonempty string is continuing to allow extensions to run. Fix that, and move
the config option parsing to extension_commands() to make sure the fix only
needs to happen once.

Fixes: #238
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00