[Whonix-devel] Whonix testing (was: Re: [qubes-devel] More regular point releases schedule?)

Mon Feb 18 09:06:00 CET 2019

Marek Marczykowski-Górecki:
> On Sat, Feb 16, 2019 at 04:12:00PM +0000, Patrick Schleizer wrote:
>> Marek Marczykowski-Górecki:>> Other prudent tests:
>>>> Check if torbrowser starts from anon-whonix and DispVM. I could add test
>>>> to "whonixcheck --test" which checks that Tor Browser indeed ended up in
>>>> user home folder and has expected hardcoded version number to automate
>>>> that as well if deemed useful.
>>>
>>>> ("--test" or so meaning "for use in automated testing after template build")
>>>
>>> Does it mean "whonixcheck --test" should be run in addition to
>>> "whonixcheck --verbose --leak-tests"? Or one contain the other?
> 
>> That's up for consideration. Both ways are ok.
> 
>> I guess an additional --test would be fine.
> 
>> Note: --test does not exist yet.
> 
>> Meanwhile "whonixcheck --verbose --leak-tests" will be a good start.
> 
> Added:
> https://github.com/marmarek/openqa-tests-qubesos/commit/433b562d7d2cc51b6c0c2bb82205badb95c436e9
> 
> Example run: https://openqa.qubes-os.org/tests/1220
> See "Logs & Assets" tab for detailed logs.
> 
> Note those tests are still scheduled manually. My plan is to
> automatically schedule them when updates are uploaded to testing
> repositories (applies to both individual packages and whole templates).
> But it isn't done yet.

Awesome!

> BTW I have a little problem with "unwanted packages" check. For
> integration tests, I install several packages in templates, needed by
> some tests. This include "python-pip" (because some python packages are
> too old or unavailable in Debian for tests to work - applies to python-uinput and
> python-dogtail). This makes "unwanted packages" check to fail.
> For now I've completely separated whonixcheck run from other tests, but
> this means I need to install templates for tests twice - which adds
> about 30min to the whole test run. Not a big deal right now.

Btw usually all functions are easily skippable. Except this one. This
will be easily skippable with the next upgrade of whonixcheck which will
be soon.

Git fix:

https://github.com/Whonix/whonixcheck/commit/7dcbf0a3e48336a88739e7ca36f3c2cccdd2d27a

Once upgraded, the way to fix it is using the usual
whonixcheck_skip_functions variable mechanism.

as root:

echo 'whonixcheck_skip_functions+=" check_unwanted_packages "' >
/etc/whonix.d/40_qubes_test.conf

(And later delete /etc/whonix.d/40_qubes_test.conf if keeping the image
perhaps.)

It may be possible to expose this variable using an environment variable.

Alternatively, we could modify the whonixcheck_unwanted_package variable.

The way /usr/bin/whonixcheck looks like now...

https://github.com/Whonix/whonixcheck/blob/master/usr/bin/whonixcheck

... I did not think through yet if it would be a good idea to pass "sudo
-E" (preserve environment).

It's added in config with this:

whonixcheck_unwanted_package+=" python-pip "

/etc/whonix.d/30_whonixcheck_default.conf contains:

## To remove selected packages from the list of unwanted packages,
## you could add something like  the following to your
## "/etc/whonix.d/50_whonixcheck_user.conf". (Replace the below example
that is
## using 'popularity-contest' with the actual name of the package you want
## to remove from the list.
#whonixcheck_unwanted_package="$(echo "$whonixcheck_unwanted_package" |
sed 's/ popularity-contest //g')"

So...

whonixcheck_unwanted_package="$(echo "$whonixcheck_unwanted_package" |
sed 's/ python-pip //g')"

... in config /etc/whonix.d/40_qubes_test.conf would do also.

> It would be probably unwise to disable "unwanted packages" check.

So perhaps modification of the whonixcheck_unwanted_package variable is
the way to go?