{{Header}} {{#seo: |description=Instructions to build {{project_name_long}} Virtual Machine Images from Source Code. }} {{title|title= {{project_name_short}} VM Build Documentation }} {{intro| Instructions to build {{project_name_short}} Virtual Machine Images from Source Code. }} = Introduction = This page documents how to build {{project_name_short}} VM (Virtual Machine) images for [[VirtualBox]] ({{Code2|.ova}}) or [[KVM]] ({{Code2|.qcow2}}) [[Trust#Builds_from_Source_Code_versus_Builds_including_Binary_Packages|"from source code"]]. Most users do not build {{project_name_short}} from source code but instead [[Download|download {{project_name_short}}]]. For [[Qubes|{{q_project_name_long}}]], see [[Dev/Qubes#Build_{{q_project_name_short}}_Templates|{{q_project_name_long}} Build Documentation]]. Building from source code as security advantages (see [[Trust]]). The goal of this build documentation is to be usable by as many users as possible. Therefore written as easy as possible with no prior knowledge of tools used required. The high level summary is: # Host preparation. # Get derivative-maker source code. # Run derivative-maker. # Done, build of {{project_name_short}} has been completed. Advanced users that already know how to use git and how to perform digital software verification using OpenPGP (gpg) do not need to strictly follow this documentation. See footnote(s) for details. For example, these instructions fetch only a specific git tag. If you know how to use git, you are of course free to git clone the whole repository and then use git verify-tag, git verify-commit, git checkout --quiet and git generally as per usual. The reason why more specific commands are being used on this page is to simplify the process for laymen, to make these instructions as fail-safe as possible. Due to digital software verification instructions and [[Software_Signature_Verification_Usability_Issues_and_Proposed_Solutions|Software Signature Verification Usability Issues]] these instructions might look more complicated then they actually are. {{always_verify_signatures_reminder}} Steps concerning digital software verification are pointed out as "This step is recommended for better security, but is not strictly required. (See [[Trust]].)" When verifying digital software signatures, these instructions will be slightly more complicated but therefore even more secure. # Host preparation. # Get {{project_name_short}} source code (which includes {{project_name_short}} build script). # Verify digital software signatures. # Run {{project_name_short}} build script # Done, build of {{project_name_short}} has been completed. = Host Preparation = == Host Preparation Notices == {{Expand_or_Collapse_All}}
* operating system: You need to build on '''Debian''' {{Stable project version based on Debian codename}}, such as {{project_name_workstation_long}} {{VersionShort}} or a Debian {{Stable project version based on Debian codename}} VM. * disk space: You need ~ 30 GB free disk space. * RAM: You need ~ 4 GB free RAM. ** Might actually work with a lot less RAM. ** Might actually work with less RAM if you have swap. * linux user account: Do not build under user root. Login regular as user user. Do not use sudo to run derivative-maker because this is handled by the build script. * build place notice: You cannot build on {{project_name_gateway_long}} (due to networking issues). * build logs: Build logs: If using a GUI (graphical user interface) it is recommended to set your terminal (for example xfce4-terminal) to unlimited scrollback, so you can watch the full build log and share it for support in case there are issues.
...
* private files in source code warning: Do not add private files to {{project_name_short}} source code folder! [...]
Unless you know what you are doing. Technically, it would work. This is recommended against. Those files would get managed by the respective package. When you later update {{project_name_short}} debian packages, your files would get deleted by the package manager. Also adding private files to {{project_name_short}} source code folder, later contributing to {{project_name_short}} development and accidentally pushing the wrong git branch would be a disaster. Better add your private files to {{project_name_short}} after building {{project_name_short}}. Or add a custom build step adding your files, which then get copied from a folder outside of {{project_name_short}} source folder. See "Source Code Changes" in "Optional Build Configuration" below.
* Exiting VMs name collision warning: Make sure there aren't any VMs in VirtualBox (inside your build machine) already called {{project_name_gateway_short}} or {{project_name_workstation_short}}!
Because the build script would fail, because it tries to create VMs either named {{project_name_gateway_short}} or {{project_name_workstation_short}}. Running the clean script between builds will prevent this error.
* Parallel builds unsupported warning: Do not try to build {{project_name_gateway_short}} and {{project_name_workstation_short}} at the same time!
Building {{project_name_gateway_short}} and {{project_name_workstation_short}} at the same time is not supported due to limitations in the build script. In other words, do not try to run for example {{Code|~/derivative-maker/derivative-maker --flavor whonix-gateway -- --target virtualbox}} and {{Code|~/derivative-maker/derivative-maker --flavor whonix-workstation -- --target virtualbox}} at the same time. The build would probably fail.
* CI warning: Do not use images created inside Continuous Integration (CI) environments for anything besides testing!
Usually you are not using [https://en.wikipedia.org/wiki/Continuous_integration CI] environments without knowing. You can find out if you are running inside a CI environment by running. {{CodeSelect|code= echo "$CI" }} If it shows nothing, i.e.

Everything is fine. Otherwise, if it were to show.
true
Then do not use these images for anything besides testing. Reason: https://github.com/{{project_name_short}}/derivative-maker/blob/master/build-steps.d/1200_prepare-build-machine
== Host Preparation Steps == '''1.''' build dependencies: {{Build_Dependencies}} '''2.''' Platform specific notices: * VirtualBox: Install VirtualBox on the build machine as per [[VirtualBox/Recommended_Version|recommended VirtualBox version]]. Due to technical challenges, see [[Dev/VirtualBox#VirtualBox_Installation_Challenges|VirtualBox Installation Challenges]]. * Other platforms: No special notice. '''3.''' sudo setup: Chose either Option A or Option B. {{Tab |type=controller |id=sudo-setup |content= {{Tab |active=true |id=sudo-setup |addToClass=info-box active |title=== Option A: passwordless sudo == |content= Option A: Set up passwordless sudo. * Abstract task overview: Setup passwordless sudo for your Linux user account name. * Detailed task overview: Setup sudo and add your current linux user account to the linux user group sudo. The following instructions use Linux user account name user as an example. If the user is using a different Linux user account name such as myname then the commands below have to be adjusted accordingly. * Unspecific: Configuration of sudo is [[unspecific|unspecific to {{project_name_short}}]]. * {{project_name_short}} specific: When building inside {{project_name_short}}, user user is already a member of group sudo by default. Therefore the following step "1." can be skipped. * Detailed steps: '''1.''' {{Sudo_Setup}} '''2. ''' Configure user user to be able to use sudo without a password. {{CodeSelect|code= echo '%sudo ALL=(ALL:ALL) NOPASSWD:ALL' {{!}} sudo EDITOR=tee visudo -f /etc/sudoers.d/dist-build-sudo-passwordless >/dev/null }} '''3.''' Done. The process of setting up passwordless sudo has been completed. }} {{Tab |id=sudo-setup |addToClass=info-box |title=== Option B: long sudo timeout == |content= Option B: Set a longer sudo password timeout. * Abstract task overview: Setup sudo with a longer timeout for your Linux user account name. * Detailed task overview: Setup sudo and increase the sudo password timeout (four all Linux user accounts or all users). The following instructions use Linux user account name user as an example. If the user is using a different Linux user account name such as myname then the commands below have to be adjusted accordingly. * Unspecific: Configuration of sudo is [[unspecific|unspecific to {{project_name_short}}]]. * {{project_name_short}} specific: When building inside {{project_name_short}}, user user is already a member of group sudo by default. Therefore the following step "1." can be skipped. * Detailed steps: '''1.''' {{Sudo_Setup}} '''2. ''' Configure user user to be able to use sudo without a password. {{CodeSelect|code= echo 'Defaults timestamp_timeout=30' {{!}} sudo EDITOR=tee visudo -f /etc/sudoers.d/dist-build-sudo-timeout >/dev/null }} '''3.''' Done. The process of setting up a longer sudo timeout has been completed. }} }} = Special Notice = There is currently no special notice. = Choose Version = {{mbox | image = [[File:Ambox_notice.png|40px|alt={{project_name_short}} default admin password is: changeme]] | text = Version Numbers Choice Information '''1.''' Common sense is required when choosing the right version number. '''2.''' For example, the latest available git tag version number is not necessarily the most stable or suitable. '''3.''' Follow the [[Stay Tuned|{{project_name_short}} News Blog]] as it might contain information. '''4.''' This documentation will be using {{VersionNew}}-stable as an example. Replace {{VersionNew}}-stable with the actual version chosen for the build. '''5.''' Git tags by convention: * have one of the following appendixes stable, testers-only or developers-only version. * For example: ** {{VersionNew}}-stable ** {{VersionNew}}-testers-only ** {{VersionNew}}-developers-only * all represent the very same source code. {{CodeSelect|code= git diff {{VersionNew}}-stable {{VersionNew}}-developers-only }} '''6.''' If there is only git tag {{VersionNew}}-developers-only available but a stable release announcement has been published, then there is no need to wait for the creation of {{VersionNew}}-stable since it actually has already been blessed stable, would contain the identical source code anyhow but just have a different git tag name. }} {{mbox | image = [[File:Ambox_notice.png|40px|alt={{project_name_short}} default admin password is: changeme]] | text = Version Numbers Choice Instructions '''1.''' Go to https://forums.{{project_clearnet}}/c/news/ and look for the latest stable version number. '''2.''' Compare with the stable version number mentioned in the wiki. '''3.''' For example, if {{VersionNew}} has been announced in the forums and is available from the wiki, git tag {{VersionNew}}-developers-only is avaialble but {{VersionNew}}-stable is not, then {{VersionNew}}-developers-only can be safely used. }} = Get the Signing Key = {{Get_Signing_Key}} = Get the Source Code = {{Build_Documentation_Get_Source}} = Change Directory = Get into derivative-maker source code folder because later on package build commands using ./derivative-maker are expected to be run from the root of the source folder. {{CodeSelect|code= cd derivative-maker }} = OpenPGP Verify the Source Code = {{OpenPGP Verify the Source Code}} = Checkout Version = {{ Build Documentation Choose Version |version={{VersionNew}}-stable |extra=--recurse-submodules }} = Check Git = {{Build Documentation Check Git |version={{VersionNew}}-stable }} = VM Creation = '''1.''' Build target selection. Mandatory The following build targets are available. Choose a --target. Either option '''A)''', '''B)''', '''C)''' or '''D)'''. Multiple build targets at the same time are also possible. {{CodeSelect|code= --target virtualbox --target qcow2 --target raw }} [[Dev/Build_Documentation/Physical_Isolation|Physical Isolation Build Documentation]] {{CodeSelect|code= --target root }} {| class="wikitable" style="margin-left: auto; margin-right: auto; border: none; text-align: center;" ! align="center" | Option ! align="center" | Build Target ! align="center" | Comment ! align="center" | Image Type ! align="center" | Target Parameter |- | '''A)''' | [[VirtualBox]] VMs | Production. | .vdi | {{CodeSelect|code=--target virtualbox}} |- | '''B)''' | [[KVM]] (and [[QEMU]]) | KVM: Production.; QEMU: [[unsupported]] | .qcow2 | {{CodeSelect|code=--target qcow2}} |- | '''C)''' | raw images | Mostly interesting for developers / porters. | .raw | {{CodeSelect|code=--target raw}} |- | '''D)''' | UTM with QEMU * [[MacOS#M1]] * https://forums.whonix.org/t/whonix-on-mac-m1-arm/11310 | Tested on Mac M1, testers only, --arch arm64. | .raw | {{CodeSelect|code=--target utm}} |- |} '''2.''' Flavor selection. Mandatory. Choose a flavor. {| class="wikitable" style="margin-left: auto; margin-right: auto; border: none; text-align: center;" ! Option ! Flavor Name ! Flavor Parameter |- | '''A)''' |{{project_name_gateway_short}} CLI |--flavor {{project_name_short_lowercase}}-gateway-cli |- | '''B)''' |{{project_name_gateway_short}} Xfce |--flavor {{project_name_short_lowercase}}-gateway-xfce |- | '''C)''' |{{project_name_workstation_short}} CLI |--flavor {{project_name_short_lowercase}}-workstation-cli |- | '''D)''' |{{project_name_workstation_short}} Xfce |--flavor {{project_name_short_lowercase}}-workstation-xfce |- |} '''3.''' CPU target architecture selection. Mandatory. Choose a target CPU architecture. {| class="wikitable" style="margin-left: auto; margin-right: auto; border: none; text-align: center;" ! Option ! CPU Architecture Name ! Comment ! CPU Architecture Parameter |- | '''A)''' | Intel and AMD64 {{Amd64}} | Default, best tested. | --arch amd64 |- | '''B)''' | i386 | [[Unsupported]]. [[Dev/64bit|32-bit or 64-bit?]] | --arch i386 |- | '''C)''' | ARM64 | | --arch arm64 |- | '''D)''' | Others. | Potentially. | * Build script parameter --arch results in setting the BUILD_TARGET_ARCH build variable. If you inspect ([[Dev/git#grep|grep tip]]) the build scripts for the variable name you can see other architectures might work but are untested. * [[Dev/Porting|porting {{project_name_short}} to other platforms]] |- |} '''4.''' {{project_name_short}} APT repository choice. Optional. Enable [[Project-APT-Repository|{{project_name_short}} APT repository]] inside the images. --repo true will set export build_remote_repo_enable="true" which results in setting
DERIVATIVE_APT_REPOSITORY_OPTS="--enable --codename $derivative-maker_apt_stable_release"
export DERIVATIVE_APT_REPOSITORY_OPTS
derivative-maker_apt_stable_release defaults to {{Stable project version based on Debian codename}} and could optionally through a [[Dev/Source_Code_Intro#Build_Configuration|build configuration]] set to {{Stable project version based on Debian codename}}-proposed-updates, {{Stable project version based on Debian codename}}-testers or {{Stable project version based on Debian codename}}-developers.
See [[Trust]]. This is done for official {{project_name_short}} [[Dev/Redistribution|redistributeable]] builds. {{CodeSelect|code= --repo true }} '''5.''' Optional configurations. Optional. See also [[Build_Configuration|Optional Build Configuration]]. '''6.''' Notices. * These instructions assume you have the {{project_name_short}} source code in your home folder (~/derivative-maker), and * use VirtualBox as an example. ---- '''7.''' Delete any existing {{project_name_gateway_short}} virtual machine with the following command. '''Warning''': This will delete any virtual machine named {{project_name_gateway_short}} from VirtualBox installed on your build machine! {{CodeSelect|code= ~/derivative-maker/derivative-delete --flavor {{project_name_short_lowercase}}-gateway-xfce --target virtualbox }} '''8.''' Delete any existing {{project_name_workstation_short}} virtual machine with the following command. '''Warning''': This will delete any virtual machine named {{project_name_workstation_short}} from VirtualBox installed on your build machine! {{CodeSelect|code= ~/derivative-maker/derivative-delete --flavor {{project_name_short_lowercase}}-workstation-xfce --target virtualbox }} ---- '''9.''' Build a {{project_name_gateway_short}} virtual machine image. {{Code2|build command}}: {{CodeSelect|code= ~/derivative-maker/derivative-maker --flavor {{project_name_short_lowercase}}-gateway-xfce --target virtualbox --repo true }} '''10.''' Build a {{project_name_workstation_short}} virtual machine image. {{Code2|build command}}: {{CodeSelect|code= ~/derivative-maker/derivative-maker --flavor {{project_name_short_lowercase}}-workstation-xfce --target virtualbox --repo true }} ---- While building, you might see a few [[Dev/Expected Build Warnings|Expected Build Warnings]]. = Build Logs = '''Optional.''' Mostly useful in case of build issues where sharing the build log is necessary for support. If using: * '''A)''' GUI (graphical user interface): It is recommended to set your terminal (for example xfce4-terminal) to unlimited scrollback, so you can watch the full build log and share it for support in case there are issues. * '''B)''' CLI (command line user interface): In case there are build issues, it is recommended to redirect the build log to a text file. To do that, for example {{CodeSelect|code= >> ~/build-log 2>&1 }} could be appended to the very end of the {{Code2|build command}} to redirect the build output to {{Code2|build log file}} ~/build-log. ** Note: when using that command there will be no output in the terminal until the build is completed. This is as per Linux default and [[unspecific|unspecific to {{project_name_short}}]]. The build might not complete. In that case, that command would hang forever. To see if the build is stuck, it would be possible to open the {{Code2|build log file}} with a text editor. In that case, the file would have to be re-opened or refreshed to with F5 key to see new build output appended to the build log file. Or by using for example {{CodeSelect|code= tail -n 100 -f ~/build-log 2>&1 }} = VM Build Result = * VirtualBox: The newly created VMs can be seen in VirtualBox user interface and in the usual VirtualBox data folders. * KVM, QEMU, raw images: The resulting .qcow2 and/or .raw images can be found in ~/derivative-binary folder. Most users have completed the build process at this point, can start using {{project_name_short}} and skip the rest of this page. = Unified Images = Optional. Creation of ([https://forums.whonix.org/t/whonix-virtualbox-14-0-1-4-4-unified-ova-downloads-point-release/6996 unified]) .ova image(s) for {{project_name_short}} VirtualBox or libvirt.xz archives for {{project_name_short}} KVM. This might only be interesting if the goal is [[Dev/Redistribution|Redistribution]] of images to third parties such as the public. There are two options. * '''A''') Automated: A bit difficult (since it expects preexisting signing keys), using [[#Prepare Release|dm-prepare-release]] script, OR * '''B''') Manually: ** VirtualBox: Using the VirtualBox graphical or command line interface VM export feature could be used. This is sane since important VM settings were already configured in https://github.com/{{project_name_short}}/derivative-maker/blob/master/build-steps.d/4600_create-vbox-vm. dm-prepare-release VM export does nothing special/important for privately used builds. ** KVM: Manually. = Prepare Release = [https://github.com/Kicksecure/developer-meta-files/blob/master/usr/bin/dm-prepare-release dm-prepare-release] is useful for: * Creation of a [https://forums.whonix.org/t/whonix-virtualbox-14-0-1-4-4-unified-ova-downloads-point-release/6996 unified] .ova image or libvirt.xz archive * Creation of torrent files. * Creation of hash sum files. * Creation of digital software signatures. ** [[OpenPGP]] (gpg) signatures ** [[signify|signify]] signatures ** [https://forums.whonix.org/t/use-codecrypt-to-sign-whonix-releases/7844 soon] {{kicksecure_wiki |wikipage=PQCrypto#Codecrypt |text=Codecrypt }} signatures * Adding the license agreement. * Adding the disclaimer. * [[Dev/Redistribution|Redistribution]]. * Example: {{CodeSelect|code= dm-prepare-release --build --target virtualbox --flavor whonix-workstation-xfce }} For private builds, i.e. for builds which are not supposed to be redistributed to others, none of this is important. If any of this was important, it could also be done manually. = Footnotes = {{reflist|close=1}} {{Footer}} [[Category:Development]]