Support a list of projects in the arguments. The listed projects will
be limited to those named, in the order they are given on the command
line. This is the same way that "fetch", "forall", and other commands
operate on all projects by default, or just specified projects.
Remove the special-case text printing the manifest path. We'll handle
that by treating west and the manifest as projects in a later patch.
Signed-off-by: Marti Bolivar <marti@foundries.io>
This creates extra slashes when users naively concatenate two paths
with the system path separator. It also looks weird on Windows.
Signed-off-by: Marti Bolivar <marti@foundries.io>
This reverts commit bc7b1c595ed4090be6d80859a01a0bb372ae030c in
the zephyr main tree to fix the runner script and also does the
below modifications.
1) explicitly terminate processes.
2) implicitly call debugserver when debug is called.
Signed-off-by: Rajavardhan Gundi <rajavardhan.gundi@intel.com>
The mimxrt1052 does not have any internal flash, therefore a reset after
load blows away the code when loaded into sram. Reverse the order of the
pyocd commands such that the load follows the reset.
The command order now matches jlink.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
'git fetch/pull' by default doesn't fetch tags from remotes specified as
URLs, which is what we use. Pass --tags to get them.
This will only fetch tags pointing to commits reachable from the
manifest revision being fetched.
Fixes: #92
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
On Windows shlex.split() will remove all backslash separators
unless specifically instructed.
Quote the path to the manifest before splitting so that
backslash path separators don't get removed on Windows.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Tests were failing on Windows due to not using normalized paths.
Use normalized paths everywhere to avoid wrong path comparisons.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
When creating the absolute path to a project in the manifest, use
os.path.normpath() to get correct path formatting on all platforms.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of fetching from remotes, fetch from URLs. This bypasses the
entire remote system, which is nice in that we no longer rely on how
remotes are configured. Users could delete remotes or point them
somewhere else, and things will still work.
I had somehow managed to miss all along is that Git allows URLs wherever
names of remotes are allowed.
Also put URLs into the initial west/config configuration file. Keep the
name 'remote' for the configuration value, as putting the name of a
remote there will work as well.
IOW, both of the following will work:
[west]
remote = https://github.com/zephyrproject-rtos/west
revision = master
[west]
remote = my-custom-remote
revision = master
For projects, create a remote named after the remote in the manifest
(like before), just for the user's convenience. We never use it
ourselves.
Fixes: #83
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add 'git@' to the list of accepted prefixes. Alternatively, any username
could be accepted, but keep it simple while we have the URL check.
IMO, the check should be removed instead. It artificially restricts us
to a subset of what Git really supports, for no good reason.
Also fix a typo: s/git-shh:/git-ssh:/
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Using `-u` as the manifest URL is confusing because:
* We use `-w` for the west url
* We use `--wr` for the west revision
* We use `--mr` for the manifest revision
Instead, use `-m` which is consistent with the rest of the options.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
As long as that's the case, it should be documented.
I suspect it might be a feature. If you've created a branch do to some
work with 'west branch fix-stuff repo1 repo2 repo3', then it's probably
unexpected if 'west checkout fix-stuff' checks out a remote branch
'fix-stuff' that happens to exist in some other repository.
Fixes: #34
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This came up in https://github.com/zephyrproject-rtos/west/issues/52.
I'm not sure what the appropriate field widths are ({:n}). We could
tweak it later if some fields turn out to be too short/long.
Fixes: #37
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
If either the west or the manifest repository has local modifications,
skip automatically updating the modified repository, with a warning.
This avoids merge conflicts when working on them.
Maybe the manifest repository update could be skipped if the west
repository is out of date, and vice versa, but I suspect this version
might be good enough.
Fixes: #59Fixes: #62
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Makes it a bit clearer what's going on, and helps people find the
configuration file.
Example output:
Initializing in /home/ulf/westwest
=== Cloning west repository from https://github.com/zephyrproject-rtos/west, rev. master ===
Cloning into '/home/ulf/westwest/west/west'...
remote: Enumerating objects: 803, done.
remote: Total 803 (delta 0), reused 0 (delta 0), pack-reused 803
Receiving objects: 100% (803/803), 239.55 KiB | 371.00 KiB/s, done.
Resolving deltas: 100% (428/428), done.
=== Cloning manifest repisitory from https://github.com/zephyrproject-rtos/manifest, rev. master ===
Cloning into '/home/ulf/westwest/west/manifest'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 25 (delta 0), reused 2 (delta 0), pack-reused 19
Unpacking objects: 100% (25/25), done.
=== Initial configuration written to /home/ulf/westwest/west/config ===
=== West initialized ===
Maybe some helpful hint could be added to the end too.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
This is just a shorthand for 'west fetch' + 'west checkout -b <branch>'.
The name of the branch is based on the revision from the manifest. For
revisions that are qualified refs (refs/heads/foo), the last component
(foo) is used as the name.
For SHA revisions, it doesn't make much much sense to name the branch
after the SHA, so 'work' is used instead.
The name of the created branch(es) can be overriden by passing
'-s <branch name>' to 'west clone'.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Just have two _update_manifest() and _update_west() functions and get
rid of the flags. Bit easier to follow.
_update() was a bit overly generic as a name too.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Add a context manager ('with foo:' thingy) for setting a context for a
block of code. The context is just some text that gets added to the
error whenever a Git command fails.
Use the context manager to point out whenever a failing Git command is
related to self-updates. That could be a bit tricky to debug otherwise.
Example error:
FATAL ERROR: Command 'git rebase 'FETCH_HEAD^{commit}'' failed for
manifest (west/manifest/), while running automatic self-update. Please
fix the state of the repository, or pass --no-update to 'west
fetch/pull' to skip updating the manifest and West for the duration of
the command.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
_update() updates the special west/ and manifest/ repositories and has
code for checking if HEAD contains all the commits from a particular
revision (is up-to-date re. the revision). Move that code into a
separate _up_to_date_with() function, and also use it when rebasing
normal projects.
This gives a less spammy message when running 'pull' and 'rebase' on
already up-to-date projects:
$ west rebase
=== zephyr (zephyr/) is up-to-date with manifest-rev
This commit also removes the special error message when self-updating
fails, returning to plain _git() with check=True. A separate context
hint mechanism will be added after this commit instead.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Store the name of the remote used for West and manifest updates in
west/config, like Git does it. That way the remote can be switched
(after adding a remote).
Another option would be to store a URL, but it's less flexible and would
add hairy remote management code to rewrite remote URLs. It'd also make
it impossible for users to manually manage remotes in Git, as their
settings would get overwritten.
We could probably get rid of the Remote.url field. That information is
already available in the Git repository (we could fetch it from there if
we want to show it), and we should respect the repository settings in
case they have been changed by the user.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is nice in case something tries to read a configuration value
before the configuration files have been read. This could happen for the
logging functions for example, and for tests.
The configuration is empty before the configuration files have been
read, so defaults must be provided. They must be anyway though, for the
code to be robust in case configuration files are missing.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Based on the corresponding git option (see git-config(1)).
'true'/'false' are the only supported values at the moment, but they
work the same as in git (output is colorized if color.ui = true and the
output file is a terminal). If color.ui isn't specified, it defaults to
true.
color.ui is not written to west/config, since it makes more sense as a
global option than a per-repository option. It can still be disabled in
west/config though, though ~/.westconfig would make more sense.
I was thinking of toggling colors at a higher level with colorama
at first, but colorama.init() makes it a bit tricky to get right for
both Linux and Windows. Probably not too bad to make it explicit like
this.
Add a _reset_colors() helper to log.py as well to factor out a tricky
and easily typo'd print().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
To be consistent with repo (though I haven't actually checked the
behavior myself).
Fixes: #73
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The old _fetch() behavior was to check out a detached HEAD at
'manifest-rev' if the repository was initialized in the same call to
_fetch(). This meant that no detached HEAD was checked out if the
initial fetch was aborted and then resumed.
Instead, always check out a detached head at 'manifest-rev' if nothing
is checked out in the repository (which is almost guaranteed to mean
that we're still in the 'git init' state). That makes aborting and
resuming safe.
Nothing being checked out can be detected by looking at HEAD, which will
point to a non-existing ref.
Fixes: #78
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Switch to a simpler and more flexible approach for dealing with upstream
revisions, that works with any revision format (SHA, branch, qualified
ref (e.g. refs/heads/foo), or (annotated) tag):
1. Fetch (just) the upstream revision, exactly as specified.
2. Dereference it to a commit via FETCH_HEAD, and point 'manifest-rev'
at it.
Using FETCH_HEAD makes it possible to apply ^{commit} peeling (see
git-rev-parse(1)), which makes things work for e.g. annotated tags
too (git tag -a ...).
A regression (could be an improvement, depending on how you see it) is
that just the upstream revision is fetched now. It'd be simple to do a
full fetch if we'd want that later though.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add configuration file support via the standard Python configparser
module, which uses a Git-like .ini format. Use it to save the west and
manifest revisions specified when running 'west init', and use the saved
revisions when updating the west and manifest repositories.
Configuration file paths are based on Git, for consistency with the
Git-like commands. See the FILES section in git-config(1). Mac OS and
Windows paths were checked by Carles Cufi with
'git config --list --show-origin'.
Add some new shorthands to _expand_shorthands() in project.py to factor
out some logic while removing the hardcoding of the west and manifest
upstream branches. The qual-* versions of shorthands give the full path
to refs (refs/heads/foo, remotes/origin/bar, etc.)
Fixes: #63Fixes: #67
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Allow stack traces for unexpected errors to propagate to the user.
Leave the CalledProcessError handling since subprocess failure is
basically a RuntimeError and can happen for many reasons that
aren't west's fault.
Fixes#79.
Signed-off-by: Marti Bolivar <marti@foundries.io>
I got tired of typing 'list-projects' all the time.
'list' might be a bit less obviously related to projects, but I don't
think it would be that bad.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
The URL is taken literally from the manifest. It's unlikely that anyone
would put a URL starting with a '-', but having weirdly named stuff be
correctly interpreted by Git could help with diagnosing bugs.
Maybe some other Git commands could be tightened up too, but they looked
safe from a glance.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This doesn't affect the default functionality of the runner, but it's
nicer to flash hex files if possible. For example, some tools find it
easier to post-process hex files than binary files for flashing on the
board.
Keep test cases up to date.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Signed-off-by: Michael Scott <michael@foundries.io>
Set ZEPHYR_BASE in the environment for the duration of the command if
possible.
In order of decreasing precedence, get ZEPHYR_BASE from:
1. The west command line
2. The manifest file project named 'zephyr'
3. The calling environment's ZEPHYR_BASE
If the calling environment's ZEPHYR_BASE differs from the value
actually used, emit a warning. The user has probably run a zephyr-env
script in another Zephyr directory and forgotten about that, and will
almost certainly not like what they get.
Hopefully over time we can deprecate the zephyr-env scripts entirely,
but that remains to be seen.
Fixes: #46
Signed-off-by: Marti Bolivar <marti@foundries.io>
While implementing a manifest-based scan for the zephyr project, I
noticed that we aren't ensuring that projects always have distinct
paths (it's generally OK for them to have non-overlapping names; they
just can't be cloned to the same directories on the file system).
Add a check for that in west.manifest, along with a regression test.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Currently, west.manifest mostly can just return a list of Project
types, each of which is a namedtuple for the project contents we care
about.
It would be better and more flexible to have a "real" data type for
the manifest, along with concrete representations for its contents
such as default values and remotes.
Reimplement the module to add just these things, adding Manifest,
Defaults, and Remote types, and re-working the Project namedtuple into
a class type. All of these (except Manifest) use __slots__ to restrict
their attributes.
Keep the current users and test cases up to date, extending the tests
a bit more as well. This moves a bit more code out of project.py into
west.manifest.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Exercise both invalid manifest detection and correct parsing of valid
manifests. To make it easier to write tests for invalid manifests,
just glob for any invalid_*.yml files in the same directory as the
test file.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Add checks for nonexistent default remotes, projects without well
defined remotes, and projects without remotes (this last one in the
absence of a default remote).
Signed-off-by: Marti Bolivar <marti@foundries.io>
pykwalify raises a CoreError if you pass it an empty manifest. This is
by definition malformed, but the current manifest code lets this
exception pass rather than catching it and reporting the issue.
Avoid the exception by explicitly checking for, and handling, empty
manifest data. This has the ancillary benefit of only loading the
manifest file once.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Add a new module for parsing the manifest, containing the Project
definition, as well as a routine for extracting projects from a
manifest. For now, this is a pretty straightforward copy/paste job,
but we do avoid calling log.die from the manifest parsing code by
adding a new MalformedManifest exception type.
Move the manifest schema to src/west/manifest-schema.yml as part of this.
Signed-off-by: Marti Bolivar <marti@foundries.io>
We are getting user feedback about missing dependencies (like
"commands" and "log") that are actually imports of internal west
modules by those names.
We've been playing games with the python path to omit typing "west",
but this always caused inconsistency (as the bootstrapper version is
imported as west._bootstrap.version) and now it's causing outright
user confusion.
Let's just be consistent and import everything as west.xxx.
Fixes#57.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The comment added to print_version_info() explains the situation:
The bootstrapper will print its own version, as well as that of
the west repository itself, then exit. So if this file is being
asked to print the version, it's because it's being run
directly, and not via the bootstrapper.
Rather than play tricks like invoking "pip show west" (which
assumes the bootstrapper was installed via pip, the common but
not universal case), refuse the temptation to make guesses and
print an honest answer.
This will go away as we phase out the west doskey macro and shell
script in the Zephyr repository.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The error message is confusing since what's output is not what was
tested. We also want to make sure to do absolute paths on both cached
and source directories just in case of issues (e.g. on Windows,
case-sensitivity is a potential source of problems).
(We can't use os.path.samefile because that's not cross-platform.)
Signed-off-by: Marti Bolivar <marti@foundries.io>
Marti Bolivar