Commit Graph

847 Commits

Author SHA1 Message Date
Marti Bolivar 4b880be54e commands: allow commands which don't require an installation
This will be useful to support init as a regular WestCommand.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar bee4547c81 commands: project: import more from os.path
Save some characters.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar ca19fed267 main.py: don't look for ZEPHYR_BASE if command is init
While this has no effect right now, it will make sense in
general (init --> no topdir --> it's not clear what zephyr you mean)
and will be required for making init a regular command.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar ce9899267a configuration: support read_config() outside of an installation
That's perfectly fine: load the system and global ones.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar 7153520647 setup.py: whitespace cleanup
Trivial.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti Bolivar 8466885412 version: set to 0.5.99
Start v0.6.0 development. We're about to delete the bootstrapper,
which is a big enough change to need a version bump.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-01 15:22:06 -06:00
Marti 1980f707d6 West v0.5.8
This adds a couple of bootstrapper error handling fixes.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti fd53d839e9 bootstrap: check for a missing west.main
Print a more helpful message in case .west/west/src doesn't contain an
importable west.main module. (A trash .west can remain if 'west init'
fails.)

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti bb718c5a78 bootstrap: check for git before running it
Avoid issues seen on Windows where git is installed but not on PATH.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-09 22:09:04 +02:00
Marti Bolivar 5113fecc09 West v0.5.7
This includes fixes for API documentation, a configuration option handling
issue, and an improved ZEPHYR_BASE check.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar f1ca4a1d11 main.py: move west config to its own section in the help
It's not about managing multiple repositories. It's a bit weird to put it on
its own, but that's better than putting it in the wrong section.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 619d64032a main: improve ZEPHYR_BASE mismatch check
Fixes: #239
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 20d1773439 commands: fix configuration option to disable extensions
We need to be using getboolean() to parse "true" and "false" strings into their
equivalent booleans. This isn't happening, so setting the option to any
nonempty string is continuing to allow extensions to run. Fix that, and move
the config option parsing to extension_commands() to make sure the fix only
needs to happen once.

Fixes: #238
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 89c8897554 commands: s/external/extension/
Change "external" to "extension" in variable names and a config option.

This configuration option is undocumented and it isn't working correctly
anyway, so changing its name won't hurt anything. The next patch will fix its
behavior.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Marti Bolivar 988376c5a0 docs and output string fixes
Fix or improve various docstrings so they are usable from Sphinx. Without this
patch, we can't provide API documentation for west from the zephyr docs tree.

Remove comments from the various schema files. We'll move this information into
the zephyr documentation.

Use "extension" instead of "external" in some log messages for consistency with
the final name of the pluggable commands that aren't built-in.

No behavioral changes other than the slightly different printlines.

Fixes: #240
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-28 19:25:19 +01:00
Torsten Rasmussen d5f8caf2c9 West v0.5.6
Release which contain a fix for the major issue in relation to
fetching.
The error could appear upon fetching when remotes are changed or force
pushed.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-20 15:32:30 -06:00
Torsten Rasmussen d98ccf9e80 git: git fetch now uses -f
It has been discovered that in the event that a different remote is
added to manifest file which contains same branches but cannot be
fast forwarded into the local refs space, then a west update will fail
with the error:
 ! [rejected] <branch> -> refs/west/<branch> (non-fast-forward)

This commit fixes this issue by introducing '-f' when fetching.

This commit has additional test for verifying the fix.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-20 15:32:30 -06:00
Marti Bolivar 7ab671189d West v0.5.5
Cut a new version for the latest changes.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-15 13:48:38 +01:00
Marti Bolivar c43c9579e4 bootstrap: clean up messaging
- Use colors (regardless of config -- this should be fixed to use config
  when we merge everything into pypi)

- Align banner output with project.py

- Don't leave a west branch checked out to avoid a warning on first
  self update

Fixes: #201
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-15 13:48:38 +01:00
Marti Bolivar 1b3dc9b596 project.py: post-refactor clean up of logging calls
While reviewing logging calls during the most recent extraction of
project.py code into manifest.py, it became clear that the colorized
and "=== '-prefixed _banner() routine was being called way too often.

Clean this up by adding a new _msg() routine that is prefixed with
"--- " and not colorized. This still lets the output stand out from
git output, but doesn't clutter the output.

Conversely, some _banner() calls were missing, such as in the Diff
class. Fix all that up and make the printing more consistent.

While we are here, normalize various project-related printlines to use
{name} as a prefix for most _msg calls related to a project. The path
was already printed where appropriate in a _banner() call that
happened earlier in these cases.

Fixes: #229
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-15 13:48:38 +01:00
Marti Bolivar 9488e57c66 More clean up for manifest.py / project.py responsibilities
This commit is further progress on #191, but doesn't fix it entirely.

West's design has always been to split code into reusable libraries
and commands which wrap them. The extra effort is justified since West
is pluggable by extension commands, which often benefit from being
able to use the library pieces.

Although the code in project.py started out self-contained, it's
gradually being refactored to fit the design.

Continue that by factoring two more features into manifest.py: helpers
for running git commands in a project and formatting strings using
project attributes.

This lets us remove almost all of the wrappers around west.log in a
way that makes the code easier to read. The remaining _inf() is
renamed _banner() to make its purpose clearer.

Delete a fair bit of dead code left over from the older multirepo
commands which were deleted from v0.4.0 while we are here.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-15 13:48:38 +01:00
Torsten Rasmussen aa8c183649 zephyr base: fixed west config zephyr.base when relative
Fixes: #231

This commit ensures that zephyr.base will be joined with west_topdir()
when a relative path is used in west config.

The previous solution was using os.path.abspath which would give
different results depending on current location when invoking west
command.

The advantage of allowing relative path in .west/config over requiring
an absolute path, is that it allows a user to rename a project's parent
folders without having to update zephyr.base.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-14 12:30:39 +01:00
Torsten Rasmussen fdb634fd8f config: Fallback to project with path 'zephyr'
This commit allows fallback to set zephyr base to point to the project
which has the path zephyr.
If zephyr base in not set in any way and no project has path 'zephyr'
then an error is printed.

Having the fallback mechanism in an independent commit allows to easily
rollback the commit when it is considered that a project always will
have zephyr base defined as part of west installation.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-12 09:33:48 -06:00
Torsten Rasmussen 701e5198de zephyr-base: ZEPHYR_BASE env takes precedence over config settings
This commit changes the order of precedence so that environment setting
takes precedence over west config option.
It also introduces a new config file setting, to allow to specify which
setting: environment or configfile that will take precedence.
To simplify search for ZEPHYR_BASE, this setting will now be written to
the local .west/config during init and selfupdate, if non-existing.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-12 09:33:48 -06:00
Torsten Rasmussen 9168e44b8a config: Fallback to configparser if configobj is missing
This commit allows fallback to python standard configparser in case
configobj package is not installed.

This allows `west config` to work with existing bootstrappers.

Having the fallback mechanism in an independent commit allows to easily
rollback the commit when it is considered that all has configobj as
part of west installation.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-11 14:59:26 -06:00
Torsten Rasmussen 397bc5bac0 command: Added `west config` command for handling of west config files
Fixes: #198 #213

This commit add the command `west config` for getting and setting of
key - value pairs in:
- Project specific: `<project>/.west/config`
- Global specific: `~/.westconfig`
- System specific, Linux:`/etc/westconfig`
                   MacOS: `/usr/local/etc/westconfig`
                   Windows: `%PROGRAMDATA%/west/config`
It also includes corresponding test cases.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-11 14:59:26 -06:00
Torsten Rasmussen 52f151b346 config: renaming of config.py into configuration.py
As part of introducing 'west config' command, the current config.py is
renamed into configuration.py in order to have distinct name for each
module as west commands have module named same as command.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-11 14:59:26 -06:00
Marti Bolivar 8aeca9dfef main.py: fix extra newline in west --help
Remove trailing whitespace from the format_help() return value. This
prevents an extra blank line from appearing in the west --help output.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-03-10 16:27:43 -06:00
Carles Cufi cc5a05fec5 bootstrap: Use normpath() to avoid initializing in wrong folder
On Windows 10, with Python 3.7.1, the following was not working:

west init -l zephyr/
or
west init -l zephyr\

because of the trailing (back)slash being interpreted as an additional
folder and west initializing itself inside the zephyr folder instead of
doing it in the enclosing one.

By normalizing the path we avoid this issue entirely.
Note that this is actually due to a regression that was fixed:
https://bugs.python.org/issue31047

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-03-05 14:26:05 +01:00
Torsten Rasmussen 786fa35647 init: Handling of missing west.yml when loading extension commands.
Fixes: #214

This commit ensures that west built-in commands are still accessible
even if west.yml cannot be found.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-28 09:54:14 +01:00
Torsten Rasmussen 2598c0585a init: fixed issue with self.path section missing in west.yml
Fixes: #218

Using empty string instead of None as None results in error when
instantiating Manifest project as special project.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-28 09:54:14 +01:00
Torsten Rasmussen 2f68481485 init: Init will allow cloning a project when ZEPHYR_BASE is set.
Fixes: #212

If ZEPHYR_BASE was set in environment, `west init` fails.
This commit ensures `west init` can still run, as long as it is
invoked outside of a west managed project tree.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-25 12:53:58 +01:00
Carles Cufi b9e8b46f47 project: Rename --no-update to --exclude-west
Since excluding west from an update procedure is limited to a single
command now (west update), clarify that in the code and use a more
explicit flag name.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-21 15:40:55 +01:00
Marti Bolivar 90f27c1bcc main.py: trivial comment fix
Delete some # characters left over from an improperly reflowed paragraph.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-21 12:20:05 +01:00
Carles Cufi f8ad2d4fb9 scripts: Update bash completion to match current set
Update the list of commands and their flags so that the bash completion
is actually useful.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-21 10:27:23 +01:00
Carles Cufi e9703d8791 tests: Add a test for the help command
A new test compares the output of -h with that of help for all
commands.
In order to be able to test commands other than the project ones,
refactor the helper functions and fixtures into a common conftest.py and
rename the enclosing folder from "project" to "commands".

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-20 13:09:57 +01:00
Marti Bolivar 64ffb67f46 main: Add help subcommand
Add a new "help" subcommand which prints the help text for any of the
existing commands, by invoking them with "-h".

Signed-off-by: Marti Bolivar <marti@foundries.io>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-20 13:09:57 +01:00
Carles Cufi 047a23d319 main: Use Popen() + exit instead of execv()
On Windows execv() is asynchronous, leading to an additional
shell being open concurrently with the existing one.
Use Popen() instead, which ensure synchronicity across all
operating systems.

https://bugs.python.org/issue9148
https://bugs.python.org/issue19124

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-20 10:21:03 +01:00
Carles Cufi 22b395c75c tox: Fix .ini file on macOS
sys.platform returns "darwin" on macOS. Fix the .ini file so that tests
actually run on macOS.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-16 08:06:25 -07:00
Andy Doan ab560b5c64 manifest: Serialize remote[url-base] properly
Since the python object is named url_base, this field is getting
frozen via as_dict with an underscore rather than the dash required
by the schema.

Signed-off-by: Andy Doan <andy@foundries.io>
2019-02-13 15:41:01 -07:00
Marti Bolivar 5d30c9c93b West v0.5.4
I screwed up the 0.5.3 upload to PyPI. Then I tried to delete it and
re-upload. This turns out to be impossible -- file names can't be
reused even if the package has been deleted.

So 0.5.3 is the point release that never was, unfortunately. 0.5.4
will be the same, but with a different number. Fun times / whoops.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-13 12:02:23 +01:00
Marti Bolivar a341db31bf West v0.5.3
This point release brings changes to the way the west repository is
updated to make it more consistent with the way projects are updated:
both now have refs/heads/manifest-rev refs which track the latest
version fetched from the remote according to the manifest.

Both are also now updated in the same way: the remote is fetched,
manifest-rev is updated to match the new upstream revision, and the
local branch is moved to a detached HEAD at that spot. The
--keep-descendants and --rebase options are available in both cases as
well in the relevant commands (selfupdate and update).

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-12 15:15:25 +01:00
Torsten Rasmussen 9a767dec7e west: West selfupdate to use same update mechanism as west update
To align the way `west selfupdate` and `west update` works then
`west selfupdate` has now been adjusted to support --rebase and
--keep-descendants to align functionality with `west update`.
This should make it easier to describe and understand the overall
update functionality in west.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-12 06:55:55 -07:00
Torsten Rasmussen 28cb52e431 list: add posixpath, like abspath but with PurePath.as_posix()
west list --format={abspath} is intended to use with Zephyr's CMake
build system. In order for CMake not escape path, such as C:\<path>
PurePath.as_posix() is used to ensure forward slashes when printing
abspath in a new posixpath format key.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 11:21:52 -07:00
Marti Bolivar 9a2a4dea40 tests: add test case for manifest --freeze
Test basic functionality when printing to stdout.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:59:44 -07:00
Marti Bolivar ed72a50563 commands: add 'manifest' command
This allows users to operate on the manifest. The only thing they can
do now is 'freeze' it by printing everything with SHAs as revisions.

This required a surprising amount of shenanigans to get the
representation to look halfway decent and reflect our desired key
ordering when printing maps. I had to incorporate some MIT-licensed
code from PyYAML itself to make it work.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:59:44 -07:00
Marti Bolivar cabb9f5e41 Maintain a manifest-rev branch in the west project too
This will aid implementing 'west manifest'. Plus it is more consistent.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:59:44 -07:00
Torsten Rasmussen 6e92ce70a7 fetch: With revision as a SHA west will fetch default ref-space
This fixes an issue where west update could fail to checkout a SHA as
revision if that SHA was located on a branch that was not being
fetched.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-06 14:40:16 -07:00
Torsten Rasmussen ad72d0b852 init: Change west init to clone manifest repo inside .west
Fixes: #186

Instead of cloning manifest repo into temp directory, which can be a
disk different from final destination, then the manifest repo is now
cloned into .west, and from there, moved to its final location.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-06 17:43:17 +01:00
Marti Bolivar bcb7635fa6 West v0.5.2.
Cut a point release to incorporate bootstrapper fixes.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-06 00:03:09 +01:00