Dev/git

From Kicksecure
< Dev
Jump to navigation Jump to search

Development Notes about git and ways to stay up to date of source code changes

Security[edit]

Definitions:

  • remote server: Means cloud server and/or third-party hosted. Not under the full control of the project.
  • release builds: Binary images such as ISO, VM images, Debian packages.

Policy:

  • Untrusted git remote servers: Git remote servers, such as GitHub, are considered untrustworthy.
  • Local release builds only: No release artifacts are produced on any remote servers. All builds are done locally.
  • Secure usage of CI: There is continuous integration, but it is for testing only. It is a "throw-away CI" only. A build is created on a remotely hosted server, but the build results (images) are not used for anything. Nothing gets touched or downloaded from the CI or any other remote server.
  • No write access for CI servers: CI servers or web servers do not have write access to source code (git).
  • Compromised git remote servers: Even if there was malicious write access to git, it would be noticed. Before making builds, the source code is always fetched, diffed, and reviewed before local merge. Only locally merged project source code that has been reviewed is used to create release builds.
  • Non-essential use of CI only: CI is only used as a "bonus", for non-essential tasks. Such as additional code scanners that are otherwise unavailable, infeasible or insecure to run locally.

See also: Trust

Repositories[edit]

Kicksecure's code is organized into a number of individual project repos, and a "master" repo for the build system that includes the other repos as submodules. Examples of individual project repos are:

The master build system repo for Kicksecure is derivative-makerarchive.org. Its primary purpose is to build all of the individual projects, and create Debian-based OS images (OVA, ISO, QCOW2, etc) with these packages pre-installed.

Individuals who are interested in building Kicksecure or auditing its code should clone the derivative-makerarchive.org repository and its submodules. Detailed instructions on building Kicksecure can be found in Dev/Build_Documentation/VM. Individuals interested in contributing to individual Kicksecure projects should clone the repository for the specific project they wish to contribute to.

Branches[edit]

Introduction[edit]

While the project is small adrelanos thinks it is best not to make a too difficult policy.

Compare[edit]

For individual project repositories, version bumps occur on every commit with the message bumped changelog version. You can find the commits corresponding to the release of the versions you are interested in, and compare them using git diff or git difftool.

Comparing the changes between two releases of Kicksecure is harder because of git limitations when working with submodules. Each release of Kicksecure is tagged, allowing you to use git checkout --recurse-submodules in the derivative-maker repository to go to a particular point in Kicksecure's development. However, the only tool in git that can show the changes between two releases of Kicksecure with submodule changes included is git diff --submodule=diff. git difftool --submodule=diff will not show you changes within submodules.

To view the output of git diff --submodule=diff in an easier-to-read format, you can use PatchViewerarchive.org to visualize the changes shown by Git. To use this:

  • Open https://github.com/megatops/PatchViewer/blob/ff5515e77c99bdad9741caebf49c87d0fbe9d1eb/PatchViewer.htmlarchive.org in your browser.
  • Download the HTML file using the download button in the upper-right corner of the page.
  • Open the downloaded HTML file in your browser.
  • In a terminal, navigate to the derivative-maker repository, and run git diff --submodule=diff old-ref new-ref > diff, replacing old-ref and new-ref as appropriate.
  • In PatchViewer, click "Choose File", and select the diff file you just generated.
  • The diff will be visualized for you as two side-by-side files.

Note that this strategy will not show detailed information about changes to binary files such as images.

master[edit]

  • Adrelanos's progress of work.
  • Not calling it unstable, because the changes will most likely never make it unstable but it is just not tested if it still builds.
  • Branching model, project readme, information, gpg...

signed git tags[edit]

Releases will be tagged and gpg signed.

Link to Source Code[edit]

https://github.com/Kicksecure/derivative-makerarchive.org

subscribe to code changes[edit]

git[edit]

Git clone githubarchive.org.

A git specific work flow could be: 

git fetch

every (few) day(s) and then git diff(tool), merge, etc.

rss feed notification[edit]

https://github.com/Kicksecure/derivative-maker/commits/master.atomarchive.org

manually in your browser[edit]

Check every now and then https://github.com/Kicksecure/derivative-maker/commits/masterarchive.org.

Search the Source Code[edit]

Introduction[edit]

greping (search for keywords inside files) or finding files the Kicksecure source code can be useful when looking for certain keywords or files. However, a common grep -r or find . will show lots of irrelevant search results due to changelog files, license files and git history. The following is a suggestions on how to securely acquire Kicksecure source code as well as on greping or finding the Kicksecure with relevant only relevant search results.

Get the Source Code[edit]

Get the Signing Key[edit]

Info This step is recommended for better security, but is not strictly required. (See Trust.)

Get the Kicksecure Signing Key and import it.

Get the Source Code[edit]

FREE

Note: By proceeding, you acknowledge that you have read, understood and agreed to our Terms of Service and License Agreement.

1. Install git.

sudo apt update && sudo apt install git

2. Get the source code including git submodules. [1] [2]

Note: Replace 17.2.8.5-stable with the actual tag you want to build.

git clone --depth=1 --branch 17.2.8.5-stable --jobs=4 --recurse-submodules --shallow-submodules https://github.com/Kicksecure/derivative-maker.git

3. Check if above command succeeded.

If there have been errors such as: 

fatal: unable to access 'https://github.com/.../': Could not resolve host: github.com

Or. 

fatal: unable to access 'https://github.com/.../': gnutls_handshake() failed: The TLS connection was non-properly terminated.

Then the download probably failed.

Checking if the download failed or succeeded can be done by checking the exit code.

Choose your shell.

Zsh in Kicksecure or Derivatives

If the last line contains something such as the following. 

zsh: exit 1

Then do not proceed and see footnote. [3]

bash and other Shells

Show the exit code.

echo $?

Output should show. 

0

If any other exit code is shown, do not proceed and see footnote. [3]

4. Done.

Git repository cloning has been completed.

OpenPGP Verify the Source Code[edit]

This chapter is recommended for better security, but is not strictly required. (See Trust.)

  • Digital signatures are a tool enhancing download security. They are commonly used across the internet and nothing special to worry about.
  • Optional, not required: Digital signatures are optional and not mandatory for using Kicksecure, but an extra security measure for advanced users. If you've never used them before, it might be overwhelming to look into them at this stage. Just ignore them for now.
  • Learn more: Curious? If you are interested in becoming more familiar with advanced computer security concepts, you can learn more about digital signatures here digital software signatures.

1. Verify the chosen tag to build.

Note: Replace with tag you want to build.

git verify-tag 17.2.8.5-stable

2. Check the output of the verification step.

If the file is verified successfully, the output will include Good signature, which is the most important thing to check. 

gpg: Good signature

This output might be followed by a warning as follows. 

gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.

This message does not alter the validity of the signature related to the downloaded key. Rather, this warning refers to the level of trust placed in the Kicksecure signing key and the web of trust. To remove this warning, the Kicksecure signing key must be personally signed with your own key.

warning Remember to check the GPG signature timestamp. For instance, if you previously saw a signature from 2023 and now see one from 2022, this could indicate a potential rollback (downgrade) or indefinite freeze attack.[4]

2. Verify the git commit to build. [5]

Note: Replace 17.2.8.5 with the actual git tag being verified.

git verify-commit 17.2.8.5-stable^{commit}

3. Check the output of the verification step.

4. Done.

Choose Version[edit]

1. Retrieve a list of available git tags.

git --no-pager tag

2. Use git checkout to select the preferred version to build.

Note: Replace 17.2.8.5-stable with the actual version chosen for the build: the stable, testers-only or developers version. Common sense is required when choosing the right version number. For example, the latest available version number is not necessarily the most stable or suitable. Follow the Kicksecure News Blog as it might contain information.

git checkout --recurse-submodules 17.2.8.5-stable

3. Digital signature verification.

Optional. If you choose to perform digital signature verification above, you could verify the currently chosen commit ("HEAD") yet again for extra security.

git verify-commit HEAD

4. Done.

Version selection has been completed.

Check Git[edit]

1. Check if you really got the version you want.

git describe

The output should show.

17.2.8.5-stable

2. Check if source folder is pristine.

git status

Output should be the following.

  • A)

HEAD detached at 17.2.8.5-stable
nothing to commit, working tree clean
or,

  • B)
Not currently on any branch.
nothing to commit, working tree clean

If it shows something else, do not continue.

3. Done.

Find Keywords[edit]

1. Create folder ~/bin.

mkdir -p ~/bin

2. Create a file ~/bin/mygrep.

3. Paste the following content.

#!/bin/bash set -x grep \ --exclude=README.md \ --exclude=GPLv2 \ --exclude=GPLv3 \ --exclude=COPYING \ --exclude=changelog.upstream-old1 \ --exclude-dir=mnt \ --exclude-dir=qubes-src/linux-template-builder/mnt \ --exclude=changelog.upstream \ --exclude-dir=".git" \ --exclude-dir=chroot-debian \ --exclude-dir=chroot-buster "$@"

4. Save.

5. Make executable.

chmod +x ~/bin/mygrep

6. Reboot required.

7. You need to be inside the source code folder. If not already, change directory.

cd derivative-maker

8. Search for a textual string (keyword).

For example grub-pc. Replace grub-pc with the actual keyword.

mygrep -r -i grub-pc

Find Files[edit]

1. Create folder ~/bin.

mkdir -p ~/bin

2. Create a file ~/bin/myfind.

3. Paste the following.

#!/bin/bash set -x find \ "$@" \ -type f \ -not -iwholename '*.git*'

4. Save.

5. Make executable.

chmod +x ~/bin/myfind

6. You need to be inside the source code folder. If not already, change directory.

cd derivative-maker

7. Search for a file name.

For example /etc/apparmor.d.

myfind . | grep --color -i /etc/apparmor.d

Compiled Code[edit]

Source code that requires complication, that gets compiled into binaries.

find . -name '*.c' -not -iwholename '*.git*' 

./sdwdate/usr/lib/sdwdate/sclockadj.c
./bindp/usr/lib/bindp.c
./kloak/src/main.c
./kloak/src/keycodes.c
./kloak/src/eventcap.c

Put folder under Git Version Control[edit]

Operating System Software and Updates

sudo apt update

Install git.

sudo apt install --no-install-recommends git

Unless you want to use git for pushing changes to remotes which you probably won't in a testing VM you can use the following git config without using any real names or pseudonyms. (These are the git suggested defaults. [6])

git config --global user.email "you@example.com"

git config --global user.name "Your Name"

Desalinize git in current folder or cd to any folder you want to put under git version control. Can even be done for /home/user folder which is very useful.

git init

Add all files to be added for next commit.

git add -A

Actually commit to git.

git commit -a -m .

Check git status of that folder.

git status

No changes registered yet to the folder since just now committed all to git. 

On branch master
nothing to commit, working tree clean

Create a test file or do some activity such as starting a browser or e-mail client.

touch test-file

Check again git status of that folder.

git status

Now git will show what changed. 

On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	test-file

nothing added to commit but untracked files present (use "git add" to track)

Goodies[edit]

Optional. Just sharing. Like it or not.

screenshotarchive.org

apt source[edit]

You need to enable deb-src in /etc/apt/sources.list.d/derivative.list! 

sudo apt update

Hit:1 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster InRelease
Hit:2 https://deb.qubes-os.org/r4.1/vm bullseye InRelease
Ign:3 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian buster InRelease
Hit:4 tor+http://5ajw6aqf3ep7sijnscdzw77t7xq4xjpsy335yb2wiwgouo7yfxtjlmid.onion buster/updates InRelease
Hit:5 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian buster Release
Ign:7 https://deb.debian.org/debian buster InRelease
Hit:8 https://security.debian.org buster/updates InRelease
Hit:9 https://deb.debian.org/debian buster Release
Hit:11 https://deb.debian.org buster InRelease
Reading package lists... Done
E: Failed to fetch tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/dists/{{Stable project version based on Debian codename}}/InRelease  Unable to find expected entry 'contrib/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Failed to fetch https://deb.debian.org/dists/stable/InRelease  Unable to find expected entry 'non-free/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.

The following messages can be ignored:

  • "Unable to find expected entry 'contrib/source/Sources'" and
  • "Unable to find expected entry 'non-free/source/Sources'"

There is nothing in these repositories and it does not limit functionality in any way. This is an inconvenient message but it will not be fixed since it was only raised once in that last five years.

apt source is fully functional. 

apt source sandbox-app-launcher

Reading package lists... Done
Picking 'kicksecure-meta-packages' as source package instead of 'sandbox-app-launcher'
NOTICE: 'kicksecure-meta-packages' packaging is maintained in the 'Git' version control system at:
https://github.com/{{project_name_short}}/sandbox-app-launcher.git
Please use:
git clone https://github.com/{{project_name_short}}/sandbox-app-launcher.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 48.9 kB of source archives.
Get:1 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (dsc) [5,513 B]
Get:2 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (tar) [37.8 kB]
Get:3 tor+http://deb.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion buster/main sandbox-app-launcher 3:7.9-1 (diff) [5,588 B]
Fetched 48.9 kB in 2s (19.4 kB/s)
dpkg-source: info: extracting sandbox-app-launcher in sandbox-app-launcher-7.9
dpkg-source: info: unpacking sandbox-app-launcher_7.9.orig.tar.xz
dpkg-source: info: unpacking sandbox-app-launcher_7.9-1.debian.tar.xz

Footnotes[edit]

  1. ↑ Optional git parameters:
    • --depth=1: Used to speed up the download.
    • --branch 17.2.8.5-stable: Usability. Used to speed up the download.
    • --jobs=4: Used to speed up the download.
    • --recurse-submodules --shallow-submodules: Usability.
    Knowledgeable git users are free to drop any of these optional parameters.
  2. ↑ Alternatively, this can be achieved with the following commands in several steps. This is useful if network issues arise.
    • Over HTTPS:
      • git clone --depth=1 --branch 17.2.8.5-stable https://github.com/Kicksecure/derivative-maker.git
    • Over SSH.
      • Might work better over slow networks but requires a GitHub account and a configured SSH public key at GitHub. See footnote [A] below.
      • git clone --depth=1 --branch 17.2.8.5-stable git@github.com:Kicksecure/derivative-maker.git
    cd derivative-maker git submodule update --init --recursive --progress --jobs=4

    [A] GitHub SSH public key setup.

    1. You need to create an SSH public key.

    Undocumented.

    2. You view your your SSH public key.

    cat ~/.ssh/id_ed25519.pub

    3. Copy it.

    4. And paste your the SSH public key on GitHub.

    https://github.com/settings/keysarchive.org

  3. ↑ 3.0 3.1 Delete the derivative-maker source code folder and retry. 
    sudo rm -r derivative-maker
  4. ↑ As defined by TUF: Attacks and Weaknesses:
  5. ↑ It is advisable to verify the signature of the git commit as well. By convention, git tags should point to signed git commits. Beginning from git tag 9.6 and above. (forum discussionarchive.org)
  6. ↑ 
    git commit -a -m .

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: empty ident name (for <(null)>) not allowed
Your support makes
all the difference!

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!

Retrieved from "https://www.kicksecure.com/w/index.php?title=Dev/git&oldid=89238"