Let's start using two spaces for our indentation.
This was chosen semi-arbitrarily to be consistent with how argparse
indents arguments and options.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
As we evolve the manifest data, we want to be able to enforce a
minimum version of west required to parse it. Add an optional
'version' key to the data to make this possible.
We have to validate this before checking the manifest against the
schema, as later versions of west will likely extend the schema in
ways that would raise errors if we used our schema to check the data.
We allow the version to be parsed as a YAML float (like 1.0) and
convert it to a string as a convenience for the user.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
It's convenient when using this API to be able to pass a string
containing YAML data instead of requiring the caller to load the YAML
data every time.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
These should be separated into paragraphs for standard library,
third-party dependency, and internal imports respectively, but they're
not. Fix that.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
GitHub user E3V3A reports that 'ProgramData' in the cygwin environment
corresponds to PROGRAMDATA in the parent environment, so we ought to
be able to use that to locate the system configuration file in a way
that is consistent with the outer native Windows shell expectations.
Fixes: #300
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
In certain situations, such as if the project remote is an empty
repository or otherwise prevents 'west update' from setting up the
manifest-rev branch, we can't freeze the manifest even though a
project has been cloned.
Handle that by raising RuntimeError, so that higher layers know what
went wrong and can print a sensible error message instead of dumping
stack.
Fixes: #314
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
We're getting increasing interest in west being its own tool, to be
used standalone from Zephyr. Unfortunately we are still too tightly
coupled for that to be possible, but the current behavior of erroring
out is not correct. Let's downgrade that to a warning and have the
warning print the current workaround.
Relates to: #246
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Add a mandatory space between shell arguments. Fix the test cases so
they actually test this as advertised.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Every time ZEPHYR_BASE and west config clashed with each other it's been
because I moved across _several_ west installations and forgot to
adjust ZEPHYR_BASE. Fix the warning message so:
- it doesn't refer to the "west config" like there's only one!
- it shows which exact west topdir it got a west config from;
- it shows the actual value in west config and not a construction.
Before:
WARNING: ZEPHYR_BASE=/home/john/west1/zephyr in the calling
environment will be used, but was set to
/home/john/west2/zephyr in west config.
After:
WARNING: ZEPHYR_BASE=/home/john/west1/zephyr in the calling
environment will be used, but the zephyr.base config option in
/home/john/west2 is "zephyr" which implies
ZEPHYR_BASE=/home/john/west2/zephyr
When something goes wrong, the user unfortunately needs the "raw" /
low-level data; synthetized information only obscures what's going on
and may even be incorrect in case of a corner case / bug.
PS: the previous message was especially confusing when the current
directory includes some symbolic link(s) hence doesn't match anything in
the warning. In that (admittedly unsupported) case, pointing straight at
the specific west config helps even more.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use the phrase "extension commands" and print the project path along
with its name.
We don't use {name_and_path} here because users probably aren't used
to that format yet.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This just prints the top level directory of the west installation.
To fit it into the 'west help' output, tweak the section headings.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
We still need to update the manifest-rev ref even if we don't
fetch. D'oh. Hotfix the issue; a follow up cleanup should separate the
update to manifest-rev from the _fetch() command and move it to
_update() only.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This allows uploading vX.Y.Z.devN development snapshots to PyPI as
needed, without having to touch version.py or go through the pull
request process.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This branch is currently the 0.7 development tree. Bump the version so
west --version shows that in the same way zephyr handles its versioning.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Except on Python 3.4, the error is not eaten -- it's still available
in the CalledProcessError's stderr attribute.
I wouldn't expect this method to print anything to the console on
errors, so let's prevent it from doing that.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Add a generic function for printing a colorized message, and
attributes which abstract the inf(), wrn(), and err() colors.
This allows commands to print using the warning color without the
WARNING: prefix, e.g.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This slipped through the cracks when adding the manifest field to each
command instance in 7694f88 ("commands: save the parsed manifest from
main"). It only shows up for extensions -- built-ins work fine.
Add a regression test.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Adjust the name of the extension command provided in the test
environment, along with some related identifiers, so they're easier to
grep for. Make the help in the yaml file consistent with the python file.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Adjust some comments and remove parentheses (which are unnecessary in
assert statements in Python).
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The west.manifest module is not in great shape:
- Though it's possible to create a manifest from data, this still
looks for a topdir and a manifest.path configuration option on
disk. This is wrong and contrary to the goal that we can parse
manifest data independently of the file system.
You can see that this doesn't really make sense in the hacks that
the west init implementation needs to make things work, and the fact
that all the tests which operate on data only have to patch
west_topdir(), etc.
- The Project definition repeatedly finds the west topdir. Even
when this is necessary, it should only be done once, and the value
cached and passed around.
- The ManifestProject abstraction makes several bizarre semantic
decisions, some of which never made sense, some of which are
just stale leftovers from when it was SpecialProject.
- The manifest_path() helper mutates the global configuration object
Add some new kwargs to the Manifest factories and constructor, which
allow the user to avoid using west_topdir() or west.configuration when
they do not want to, and propagate the new information down into
Project and ManifestProject.
This allows users to parse manifest files and data without forcing
them to be inside an actual installation (and/or to parse other
manifest files than the one currently pointed to by manifest.path), at
the cost of not having abspath or posixpath attributes in some cases.
Fix up Project by taking a new topdir kwarg and saving it in a slot.
Fix up ManifestProject so that it generally makes more sense. This now
enforces an existing requirement in the documentation that no project
can be named "manifest", as that's reserved for the manifest
repository. Fix some other thinkos as well by deprecating the revision
and url kwargs, which are SpecialProject leftovers, and taking the
path as an argument as-is, even though that means it can be None now.
I've tried to preserve backwards compatibility whenever possible,
especially in situations where the user just wants to parse the
manifest in the current west installation based on file system
searches and configuration options.
The additional kwargs to Project and ManifestProject are breaking
changes for any code which was using this module for something else,
but the lack of sense that those made before probably meant such
code didn't do anything useful, so that's okay.
This also breaks compatibility with "west list zephyr" when run
outside of the top level directory -- you need to say "west list
manifest" or give the complete relative path to zephyr now.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Add a topdir argument to each of these functions:
- read_config()
- update_config()
- delete_config()
If set, then topdir/.west/config will be used as the local
configuration file if necessary. This allows users of this module to
decouple themselves from west.util.west_dir() without having to resort
to temporarily setting environment variables.
Keep the tests up to date, adding a couple of test cases to make sure
the new kwarg works as expected.
While we're here, re-work the fixture setup so we don't ever have to
monkey-patch west.configuration._location() by always pointing the
WEST_CONFIG_xxx environment variables inside the tmpdir we're running
each test case from. This lets us avoid some extra work in tox.ini and
makes the resulting test code easier to read, without sacrificing any
safety in terms of touching the user's actual files.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
It's not necessary. It makes perfect sense to use west config --global
outside of any installation, for example.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The user-visible features in this point release are:
- an optimized west update via its new --fetch command line flag and
update.fetch config option. The default value, "smart", skips fetching
SHAs and tags which are available locally.
- better error-handling in the manifest, diff, status, forall, and
update commands.
- the west list command now accepts uncloned projects unless its
format string requires data that must be obtained from the project
repository.
- multi-repo commands now accept partial SHAs in more cases. For
example, "west init --mr SHA_PREFIX" works now.
The developer-visible additions to the west APIs are:
- west.log.banner(): new
- west.log.small_banner(): new
- west.manifest.Manifest.get_projects(): new
- west.manifest.Project.is_cloned(): new
- west.commands.WestCommand instances can now access the parsed
Manifest object via a new self.manifest property during the
do_run() call. If read, it returns the Manifest object or
aborts the command if it could not be parsed.
- west.manifest.Project.git() now has a capture_stderr kwarg
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
By default, skip fetches of revisions that resolve to SHAs or tag
names that are already available locally.
The user can override by using:
- west update -f always.
- a new update.fetch config option
Fixes: #274
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The current way we handle errors when a command means an operation has
to be done on multiple projects (such as "west diff", "west update",
etc.) is unfortunate and a bit inconsistent.
In particular, "west update" tries in some cases to keep going and
update as many projects as possible when a rebase fails, but "west
diff" will abort on the first failure. Even worse, "west update" calls
the _git() helper in various places, which means that those commands
failing halts the entire update, leaving *only* failed rebases to
allow the process to continue.
Other commands, like "west forall", don't capture errors at all, so
the failure of a subcommand does not result in the overall process
failing. Whoops.
Let's clean up this mess by adding a helper function,
_handle_failed(), to our shared parent class, which makes sure that if
any error occurred, the whole command fails. Use it from diff, status,
update, and forall.
Change the command implementations as necessary so they never call
_git(), which allows us to roll up errors from anywhere in the command
procedure without aborting the program immediately. This lets us
inline quite a bit of code while we're here.
The lone exceptions in this file are init, list, and selfupdate.
The first two still fail immediately on the first error; the final one
continues to scream and die.
I think at this point I'm willing to say the manifest.py vs project.py
responsibilities are cleaned up:
- west.manifest.Project's git helpers are for general purpose and nondestructive
use.
- west.commands.project's use of Project and its git helpers are to do
destructive operations, but also invoking the nondestructive ones
- multi-project git operations have to be handled outside of
west.manifest, e.g. in _ProjectCommand
So:
Fixes: #191
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The control flow when there is an unknown project is wrong: this is
calling is_cloned() on None. Indent it properly and add a regression
test.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Now that we have a bit more information on what happens when uncloned
projects are in the mix, handle the RuntimeError introduced in the
previous patch.
Clean up a stale comment and improve the help text while we're here.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
These are also from code in project.py. It will be useful to have
similar behavior for extensions available without duplicating code, in
case the behavior changes.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Use self.manifest.get_projects() instead of the older implementation
of similar functionality. With the previous modification to west list
in place, no behavioral changes are expected.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
That's unnecessary for almost all the format strings, and users may
want to get information about projects they don't have cloned in order
to decide whether or not to clone them.
Use the fact that we're delaying evaluation of all git related
commands to exit out only if the sha of an uncloned project is
requested. Otherwise, all the information we need is in the manifest.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This method allows finding a list of projects by name or path.
It's inspired by a helper method in projects.py, but has slightly
different semantics.
We'll move projects.py over to using it in the next patch, but having
this in manifest.py makes it API which is usable by extension commands.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This is a generally useful thing to know, and, like all the other
Project.git() wrappers, a nondestructive operation on the project
itself.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Just a cleanup we ought to do while we're refactoring in here.
Use util.canon_path(path), which is normcase(abspath(path)), instead
of normcase(realpath(path)) (in one case) or reimplementing the same
functionality (in a couple of others).
Regarding realpath:
- there's no difference on Windows (where abspath
and realpath are the same as symlinks are generally unsupported
there).
- on Unixes, avoiding realpath() means that symbolic links are not
derefenced, which is something we're trying to achieve in west in
general to support symlinks better.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Just a refactoring:
- remove exclude_manifest kwarg from _ProjectCommand._projects(),
using isinstance(..., ManifestProject) in the only user instead
- rename some variables
No functional changes expected, but this will enable moving the code
elsewhere, to make it available to extension commands which also want
similar functionality.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Now that we have a self.manifest attribute which returns the manifest
when it's available and fails gracefully when it isn't, we can avoid
re-parsing the manifest in the project command implementations.
Note this change is more convenient to make after having introduced
_ProjectCommand to allow accessing per-instance state from each of
these commands, instead of using standalone module-level functions.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This allows individual commands to avoid re-parsing. Properly
initialize it in the constructor (and topdir while we're here).
Use a property to get and set the manifest to allow us to fail with a
consistent error message when commands that require a manifest don't
get one.
We are basically saving the manifest similarly to topdir.
This opens up opportunities to avoid re-parsing the manifest
repeatedly.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
I find the _add_parser() and _project() helper methods a bit
opaque.
- The parser boilerplate being avoided could be made easier to read by
being written in a more imperative style
- the project related helpers feel more like common instance
methods (since they need the WestCommand instance or knowledge about
its arguments to do their job) than true standalone functions.
Add a ProjectCommand superclass with helper methods for parsing and
doing some of the project related functionality instead of having
separate helper functions. This will pay dividends in subsequent
patches when we need to access more instance state.
We leave some other module-level functions, like _fetch(), where they
are for now to minimize changes.
Fix up tweak various pieces of help text while we're here -- there
were some really old mistakes left lying around from the 0.3 and 0.4
days. Yikes.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
The diff and status commands with extra arguments never worked.
Remove the invalid test cases for them. The next patch is going to
fail this sort of invocation with an error instead of silently
ignoring it.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Rename _is_sha() to _maybe_sha() -- it's technically possible to have
commit refs which are branches that look like SHAs. Any branch name
that begins with a letter A-F an contains only hexadecimal characters
will work.
Allow revisions which are SHA prefixes. This makes it convenient to do
commands like "west init --mr some-sha-prefix" without the remote git
server barfing on an unknown ref.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Marti Bolivar