West, Zephyr's meta-tool
Go to file
Marc Herbert 3260b412a9 Use 'backslashreplace' not to crash on malformed UTF from subprocess
Giant commit
https://github.com/zephyrproject-rtos/hal_nxp/commit/f9f0944bc2b4fce
"Update to SDK 2.14" added files with malformed UTF-8, more precisely
with the DEGREE SIGN (°) encoded in 8bit latin1/CP1252. Maybe others.

This crashes `west grep`.

Before this fix:

```
nxp$ west grep 'TEMPERATURE_CONV_FACTOR.*Will give'

Traceback (most recent call last):
  File ".local/bin/west", line 33, in <module>
    sys.exit(load_entry_point('west', 'console_scripts', 'west')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "west/src/west/app/main.py", line 1085, in main
    app.run(argv or sys.argv[1:])
  File "west/src/west/app/main.py", line 244, in run
    self.run_command(argv, early_args)
  File "west/src/west/app/main.py", line 503, in run_command
    self.run_builtin(args, unknown)
  File "west/src/west/app/main.py", line 611, in run_builtin
    self.cmd.run(args, unknown, self.topdir,
  File "west/src/west/commands.py", line 194, in run
    self.do_run(args, unknown)
  File "west/src/west/app/project.py", line 1765, in do_run
    completed_process = self.run_subprocess(
                        ^^^^^^^^^^^^^^^^^^^^
  File "west/src/west/commands.py", line 325, in run_subprocess
    return subprocess.run(args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 2146, in _communicate
    stdout = self._translate_newlines(stdout,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1086, in _translate_newlines
    data = data.decode(encoding, errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 385: invalid start byte
```

After this fix, no crash and no interruption and:

mcux/mcux-sdk/middleware/issdk/sensors/fxpq3115_drv.h:#define FXPQ3115_TEMPERATURE_CONV_FACTOR (256) /* Will give \xb0C */
mcux/mcux-sdk/middleware/issdk/sensors/mpl3115_drv.h:#define MPL3115_TEMPERATURE_CONV_FACTOR (256) /* Will give \xb0C */

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-20 10:43:13 -08:00
.github/workflows treewide: make v3.8 the minimum supported Python 2022-08-31 16:06:42 -07:00
src/west Use 'backslashreplace' not to crash on malformed UTF from subprocess 2023-12-20 10:43:13 -08:00
tests manifest: Add a new optional description field 2023-09-05 08:59:50 -07:00
.gitignore .gitignore: add .dir-locals.el 2020-08-14 14:44:52 -07:00
LICENSE Add setuptools integration. 2018-06-06 12:21:14 -04:00
MAINTAINERS.rst MAINTAINERS.rst: updates 2023-02-21 08:53:50 -08:00
MANIFEST.in add typing information 2023-02-14 18:11:41 -08:00
README.rst README.rst: add pytest examples and mention tox limitations 2023-09-01 10:27:52 -07:00
setup.py add typing information 2023-02-14 18:11:41 -08:00
tox.ini tox.ini: set flake8 max-line-length to 100 2023-09-01 13:58:57 -07:00

README.rst

This is the Zephyr RTOS meta tool, ``west``.

https://docs.zephyrproject.org/latest/guides/west/index.html

Installation
------------

Using pip::

  pip3 install west

(Use ``pip3 uninstall west`` to uninstall it.)

Basic Usage
-----------

West lets you manage multiple Git repositories under a single directory using a
single file, called the *west manifest file*, or *manifest* for short.
By default the manifest file is named ``west.yml``.
You use ``west init`` to set up this directory, then ``west update`` to fetch
and/or update the repositories named in the manifest.

By default, west uses `upstream Zephyr's manifest file
<https://github.com/zephyrproject-rtos/zephyr/blob/main/west.yml>`_, but west
doesn't care if the manifest repository is zephyr or not. You can and are
encouraged to make your own manifest repositories to meet your needs.

For more details, see the `West guide
<https://docs.zephyrproject.org/latest/guides/west/index.html>`_ in the Zephyr
documentation.

Example usage using the upstream manifest file::

  mkdir zephyrproject && cd zephyrproject
  west init
  west update

What just happened:

- ``west init`` clones the upstream *west manifest* repository, which in this
  case is the zephyr repository. The manifest repository contains ``west.yml``,
  a YAML description of the Zephyr installation, including Git repositories and
  other metadata.

- ``west update`` clones the other repositories named in the manifest file,
  creating working trees in the installation directory ``zephyrproject``.

Use ``west init -m`` to specify another manifest repository. Use ``--mr`` to
use a revision to inialize from; if not given, the remote's default branch is used.
Use ``--mf`` to use a manifest file other than ``west.yml``.

Additional Commands
-------------------

West has multiple sub-commands. After running ``west init``, you can
run them from anywhere under ``zephyrproject``.

For a list of available commands, run ``west -h``. Get help on a
command with ``west <command> -h``.

West is extensible: you can add new commands to west without modifying its
source code. See `Extensions
<https://docs.zephyrproject.org/latest/guides/west/extensions.html>`_ in the
documentation for details.

Running the Tests
-----------------

First, install tox::

  # macOS, Windows
  pip3 install tox

  # Linux
  pip3 install --user tox

Then, run the test suite locally from the top level directory::

  tox

You can use ``--`` to tell tox to pass arguments to ``pytest``. This is
especially useful to focus on specific tests and save time. Examples::

  # Run a subset of tests
  tox  --  tests/test_project.py

  # Debug the ``test_update_narrow()`` code with ``pdb`` (but _not_ the
  # west code which is running in subprocesses)
  tox  --  --verbose --exitfirst --trace -k test_update_narrow

  # Run all tests with "import" in their name and let them log to the
  # current terminal
  tox  --  -v -k import --capture=no

The tests cannot be run with ``pytest`` directly, they require the tox
environment.

See the tox configuration file, tox.ini, for more details.

Hacking on West
---------------

This section contains notes for getting started developing west itself.

Editable Install
~~~~~~~~~~~~~~~~

To run west "live" from the current source code tree, run this command from the
top level directory in the west repository::

  pip3 install -e .

This is useful if you are actively working on west and don't want to re-package
and install a wheel each time you run it.

Installing from Source
~~~~~~~~~~~~~~~~~~~~~~

You can create and install a wheel package to install west as well.
The `wheel`_ Python package is required to do this. See "Installing Wheel"
below if you need to do this.

To build the west wheel file::

  # macOS, Linux
  python3 setup.py bdist_wheel

  # Windows
  py -3 setup.py bdist_wheel

This will create a file named ``dist/west-x.y.z-py3-none-any.whl``,
where ``x.y.z`` is the current version in setup.py.

To install the wheel::

  pip3 install -U dist/west-x.y.z-py3-none-any.whl

You can ``pip3 uninstall west`` to remove this wheel before re-installing the
version from PyPI, etc.

Installing Wheel
~~~~~~~~~~~~~~~~

On macOS and Windows, you can install wheel with::

  pip3 install wheel

That also works on Linux, but you may want to install wheel from your
system package manager instead -- e.g. if you installed pip from your
system package manager. The wheel package is likely named something
like ``python3-wheel`` in that case.

.. _wheel: https://wheel.readthedocs.io/en/latest/