Rename BadExternalCommand to ExtensionCommandError and make it a
subclass of CommandError so that it fits naturally in the hierarchy
and provides a returncode.
Add a hint attribute to it so that we can print further diagnostic
information to the user when things go wrong.
While we are here, use traceback.print_exc() in the error handlers at
main level to make it more clear that the verbosity is not altering
the control flow of the program, just what is being printed. Also use
the returncode attribute in a CalledProcessError since that is
available.
Print the entire command invocation that caused the problem with -v
added instead of making the user guess what to do.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The restriction that extension commands have names which are valid
Python identifiers is arbitrary and unnecessarily restrictive.
For example, it should be possible to have a command with a dash (-)
in its name to separate words, like "submit-pr". This is currently not
possible.
Resolve that by generating module names from an infinite iterator of
fresh per-file values instead of relying on the command name. We don't
actually need the name! Just the canonical path of the file which
defines it is important to avoid double imports.
Signed-off-by: Marti Bolivar <marti@foundries.io>
This is leftover work that we meant to do. It fixes up ugly warnings
in case the extension command is badly formed. The user can still get
a full stack trace with verbose operation enabled.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The base west update is sufficient to get work done, but it has
inconveniences:
1. When working on a local topic branch in a project, I may want to
"west update" my local installation to sync with upstream, but not
leave behind my topic branches if they are still up to date
Make this possible by adding a -k / --keep descendants option,
which leaves the checked out branch unaltered (including any
uncommitted state in the working tree) if it's a descendant of the
new manifest-rev.
2. Similarly, having to run "git rebase" by hand when updating can be
annoying, especially if the rebases would have all succeeded and
there are multiple projects with topic branches open.
Avoid this when users explicitly opt in by adding a --rebase
option, which attempts to rebase any project branches onto the new
manifest-rev.
If both -k and -r are given and applicable to a project, -k wins.
Since the west update output is much longer now, refactor the code and
make some of the warning messages less verbose.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Rather than try to use the same argument parser for figuring out which
extension command to run and actually creating it and parsing its real
arguments, just recreate the top level parser and re-parse the
original argv when handling extension commands.
This makes things like 'west -v SOME_EXTENSION_COMMAND' work again.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The behavior of 'west list SOME_INVALID_PROJECT' (and any other
commands which try to match SOME_INVALID_PROJECT with real project)
is incorrect when called from a project directory: the current
project is found instead of an error being returned.
Fix this by requiring that, when treated as a path, the given argument
exactly normalizes to a project directory. Otherwise, it is possible
for it to match nonexistent files inside of the existing project
directory.
Add some more testing. The ones that expect a CalledProcessError fail
without this project.py patch. With it, they all pass.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Generalize CommandContextError with a general purpose superclass.
Add a returncode attribute that main can use to pass to sys.exit().
This will be used in later patches.
Signed-off-by: Marti Bolivar <marti@foundries.io>
We're seeing warnings when people have ZEPHYR_BASE set to the
"<directory>" they pass to "west init -l <directory>".
Avoid them exactly in the situation that <directory> is a zephyr
repository by using the manifest directory as ZEPHYR_BASE. For
upstream, that is correct anyway.
Merge a couple of lines while here as a cosmetic improvement.
Fixes: #167
Signed-off-by: Marti Bolivar <marti@foundries.io>
This commit cleans up the second fetch, which potentially could have
changed content since first fetch.
Normal ref-space (similar to git clone) and user specified revision is
now fetched in same command.
This further allows to checkout tags in detached HEAD while still
referring to the tag name instead of FETCH_HEAD.
This commit also allows a user to specify a sha.
When specify a sha as revision, that sha must be valid in to the normal
ref-space.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Changing `west init` into using
- git init
- git fetch
- git checkout
as this allows users to specify other refs, such as pull/<no>/head
which is not possible with git clone on github.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
For various reasons, we don't load extension command modules until the
user has requested that we run a particular extension command.
As is, this limits the "west -h" output to the extension command names
and the projects which define them, which isn't as nice as what
built-in commands get.
Allow extension commands to provide help text by extending the schema
for each command entry with an optional "help" key, and display it in
the output. If it is missing, a default is provided which recommends
just running "west $EXTENSION_COMMAND_NAME -h".
Fix the help formatting by adding a colon after each extension command
name to match the style used for built-ins while we are here.
Keep test cases up to date.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Cut release 0.5.0. This is the initial tree provided to the wider
Zephyr developer community during the Zephyr v1.14 development cycle.
Signed-off-by: Marti Bolivar <marti@foundries.io>
This commit fixes:
- Issue if multiple projects defines the same command.
This will print a warning for project/command not being available
- Fixed issue where same command could be listed multiple times
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
The current output looks kind of silly. Tweak it and add a TODO
comment for further noncritical improvements.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Add the python file's directory to sys.path, to make it easier to
import other modules in the same directory.
Signed-off-by: Marti Bolivar <marti@foundries.io>
These are moving back to the zephyr repository as west extension
commands. This will (finally) let us remove the copy of west inside
zephyr to make those features available, and lets us keep a separate
west repository for bootstrapping and multirepo management while still
making it easier for people to just edit their runners.
Since this just leaves project and hidden commands, we don't print
indented groups of built-in commands, as that would be ugly and not
make sense.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Added tests to ensure missing extension files (in case a project in
not cloned) will not cause west built-in commands to fail.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
On windows a couple of manifest test cases were failing due to path
delimiter issues.
This commit fixes those test cases by using os.path.sep instead of '/'
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
This commit creates additional tests for 'west init -l' to ensure
behaviour when initializing west around an existing manifest
repository.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Ability to initialize west in a folder which contains a manifest
project (as example by an earlier 'git clone') but which is in west
context uninitialized.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Removed url and revision from west config file from west config file as
they are no longer needed when manifest repository can be handled
directly with git
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
New update command will clone, fetch, and checkout a detached HEAD
according to manifest file.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Renamed 'west update' to 'west selfupdate' to keep it separated from
future update command.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Add logic to retrieve external command specifications grouped by the
project which provides them. Filter these so built-in commands never
get shadowed, and wire them up to the argument parsing and help
printing logic. Add a special-case command handler for these which
re-parses command line arguments after instantiating the command
object and installing its parser.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Add support for a new optional 'west-commands' key in each project in
a manifest. This specifies the location of a file which describes
additional west commands provided by that project.
Extend the Manifest class to detect this key and record the
information inside. Nothing else is done about external commands in
that class, since its purpose is just to parse the manifest.
Extend west.commands with a new external_commands() function, which
consumes a Manifest and returns a list of "descriptors" of external
commands the manifest's projects declare. These descriptors contain
metadata about the commands, but do not actually import any Python
modules related to them or otherwise try to instantiate the relevant
WestCommand instances.
This is for two reasons:
1. Performance: don't import what you might not need
2. Security: don't import random code you downloaded somewhere on the
internet! Importing a module you got from somewhere random is
basically curl | sh.
This infrastructure will be wired into main() in a subsequent patch.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Some utilities which use subcommands break them into groups in their
help output, to make it easier for the reader to find what they are
looking for. Two notable examples are docker and git.
Do the same thing for west, by tweaking the COMMANDS type to group
related commands together under a string key which describes the
group, and adding a custom argument parser subclass which uses this
information when formatting the top-level command help.
Add a group of "hidden" commands which is just PostInit for
now. Eventually that will be going away, and in the meantime, we'd
like to sweep it under the rug.
I would have preferred to do this by hooking into argparse itself, but
it is annoyingly opaque about what constitutes stable API.
The top-level "west -h" output now looks like this (without ">"s):
> $ west -h
> usage: west [-h] [-z ZEPHYR_BASE] [-v] [-V] <command> ...
>
> The Zephyr RTOS meta-tool.
>
> optional arguments:
> -h, --help show this help message and exit
> -z ZEPHYR_BASE, --zephyr-base ZEPHYR_BASE
> Override the Zephyr base directory. The
> default is the manifest project with path
> "zephyr".
> -v, --verbose Display verbose output. May be given multiple
> times to increase verbosity.
> -V, --version print the program version and exit
>
> west commands used in various situations:
> building and running zephyr:
> build: compile a Zephyr application
> flash: flash and run a binary on a board
> debug: flash and interactively debug a Zephyr
> application
> debugserver: connect to board and launch a debug server
> attach: interactively debug a board
>
> managing multiple repositories in the installation:
> list: print information about projects in the west
> manifest
> clone: clone remote projects into local installation
> fetch: "git fetch" changes from project remotes
> pull: "git pull" changes from project remotes
> rebase: "git rebase" local projects onto manifest
> versions
> branch: create a branch in one or more local projects
> checkout: check out a branch in one or more local
> projects
> diff: "git diff" for one or more projects
> status: "git status" for one or more projects
> update: update the manifest and west repositories
> forall: run a command in one or more local projects
>
> Run "west <command> -h" for help on each command.
The command-level help (e.g. "west build -h") is not changed.
Fixes: #107
Signed-off-by: Marti Bolivar <marti@foundries.io>
This is a baby step on issue #107 (Improve `west --help` formatting).
Add per-command help output to each command. While we're here, rename
'init' to 'post-init' to avoid a name clash with the init command
handled by the bootstrapper.
The "commands:" help output now looks like this in west -h:
positional arguments:
<command>
build compile a Zephyr application
flash flash and run a binary on a board
debug flash and interactively debug a Zephyr application
debugserver connect to board and launch a debug server
attach interactively debug a board
post-init finish init tasks
list print information about projects in the west manifest
clone clone remote projects into local installation
fetch "git fetch" changes from project remotes
pull "git pull" changes from project remotes
rebase "git rebase" local projects onto manifest versions
branch create a branch in one or more local projects
checkout check out a branch in one or more local projects
diff "git diff" for one or more projects
status "git status" for one or more projects
update update the manifest and west repositories
forall run a command in one or more local projects
It would be nice to be able to group these subparsers and hide the
post-init command, though, which we can't seem to do with plain
argparse.
Signed-off-by: Marti Bolivar <marti@foundries.io>
In order to add support for extension commands, we will need to know
where the manifest is before we parse any command line arguments.
This means the -m argument supported by the existing project commands
has to go away. We'll always let the manifest module fetch the
contents from the location specified in the configuration file.
(The Manifest class is still capable of parsing manifests from
arbitrary file system paths.)
While we're here, add a validity check for the sections parameter to
the Manifest constructor and factory methods. This wasn't being used
properly: it should be a list. The fact that strings are sequences
means it basically gets silently ignored when passing a string.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Let's not deal with META_NAMES anymore, which is only used once. We
can emit a nicer error message for the user without it.
Signed-off-by: Marti Bolivar <marti@foundries.io>
- Fix comments and project help messages that refer to the manifest
as a distinct special repository
- Remove extra code that assumes multiple meta projects may exist
Signed-off-by: Marti Bolivar <marti@foundries.io>
Calling bootstrap() from the WestNotFound exception handler in init()
is making the WestNotFound show up in stack traces for errors in
bootstrap(), which makes for confusing output. Fix that by moving the
call to bootstrap() out.
Signed-off-by: Marti Bolivar <marti@foundries.io>
A git clone of manifest repository into temporary folder on windows
will contain read-only files.
This will cause shutil.rmtree to fail and abort bootstrapping.
Current fix is to ignore such errors on windows and rely on windows
temp folder cleanup.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
For better handling of errors the Manifest class now raises a
MalformedConfig exception if required settings are missing in the west
config file
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Topdir marker '.west_topdir' removed.
West now determines the topdir by looking for the '.west' folder.
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Added west init command to west.main to support post-process of
west init.
Added --use-cache to limit network use when cloning manifest project
repository
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
- self key introduced in manifest file to support local path location
for the repository containing the manifest file
- Removed manifest repository under west folder
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
As part of moving manifest file inside a project repository, the west
folder is renamed to .west as it will no longer contain a manifest
repository (and potentially also not west repo in future)
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
As part of moving manifest file inside a project reposity, the manifest
file is renamed from default.yml to west.yml
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Marti Bolivar