1. DEBIAN GNU/LINUX

"Encryption works. Properly implemented strong crypto systems are one of the few things that you can rely on. Unfortunately endpoint security is so terrifically weak that NSA can frequently find ways around it." (Edward Snowden)

1.01 System Hardening

System Hardening
Security Guideliness
EFF - Surveillance Self-defense	Anonymous Planet - The Hitchhiker’s Guide (PDF)
Whonix - Essential Host Security	Madaidan's Insecurities
Whonix - System Hardening Checklist	Kicksecure - Documentation
Hardened GNU/Linux - Security-focused Linux distribution	Debian - Security Management
CIS Benchmark - Debian Linux Guides	OWASP - No more insecure software
NIST.gov - Cybersecurity Framework	NSA.gov - Cybersecurity Advisories & Guidance (GitHub)
Security Tools and Utilities
Cryptomator - Put a lock on your cloud	Duplicati - Store securely encrypted backups on cloud storage services!
BleachBit - Clean Your System and Free Disk Space	Virus Total - Free virus, malware and URL online scanning
Security Enhancements and Hardening
Grsecurity© - Linux kernel security enhancements	LKRG - Linux Kernel Runtime Guard
Linux Hardened Kernel	Kernel Hardware Vulnerabilities
SDWDate - Secure time synchronization	Ram Wipe - Secure RAM wiping utility
Debian Microcode Updates	GCC - GCC Mitigation
utls - Low-level networking library	Vanguards - Tor network protection tool
Sandboxing and Security Profiles
Bubblewrap - Lightweight Linux namespaces	Kicksecure - Apparmor Profile
SystemD Sandboxed Services - Services sandboxing	Kicksecure security-misc - Security enhancements for Debian
gVisor - Sandboxed container runtime	OSfooler-ng - OS fingerprinting evasion tool
Vuurmuur - Firewall management tool	Advanced Policy Firewall - Firewall for Linux
LibreSSL - Version of the TLS/crypto stack forked from OpenSSL	Jitterentropy RNGd - High-quality entropy source for Linux
Haveged - Entropy harvesting daemon	Whonix - Anon apps config
Kloak - Keystroke anonymization tool	Wayland - Modern display server protocol
USBGuard - Tool for protecting against USB threats	Hardened Malloc - Memory allocator with security enhancements
Macchanger - MAC address randomization tool	Linux Kernel Hacking Guides

1.02 Essential Tools

Ventoy	Secure Boot Checksums	Qubes	Secure Boot Checksums
rEFInd	Secure Boot Checksums	Kali Linux	Secure Boot Checksums
Clonezilla	Secure Boot Checksums	GnuPG	Guides Check
GParted	Secure Boot Checksums	WinPE	Sergei Strelec Checksums
ShredOS	Secure Boot Checksums	Hiren's BootCD PE	Secure Boot Checksums
Tails	Secure Boot Checksums	FreeDOS 1.3 LiveCD

Others: Security Onion, NST, Android-x86, HardenedBSD, OSGeoLive, OpenWRT, LibreELEC.tv, SteamOS.

¹Check hash with GtkHash (GUI)

Check hash with GtkHash (GUI) https://gtkhash.org

$ sudo apt install gtkhash

Copy Copy

²How to summarize iso file with GNU Coreutils (CLI)

Summarize iso file with GNU Coreutils (CLI) https://gnu.org/software/coreutils/manual/html_node/Summarizing-files.html

³Manual method with sha256sum

The SHA-256 checksum hashes in a file called SHA256SUMS in the same directory listing as the download page.

First install it

$ sudo apt install sha256sum

Copy

Open a terminal and go to the correct directory to check a downloaded iso file:

$ cd download_directory

Copy

Then run the following command from within the download directory.

$ sha256sum name.iso

Copy

sha256sum should then print out a single line after calculating the hash:

sdd31231c0421be56f39c7a31245c423fgcc3b048ds321a3e83d2c4d714fa9a76 *name.iso

Compare the hash (the alphanumeric string on left) that your machine calculated with the corresponding hash in the SHA256SUMS file.

⁴Semi-automatic method with sha256sum

First download the SHA256SUMS and SHA256SUMS.gpg files to the same directory as the iso. Then run the following commands in a terminal.

$ cd download_directory

Copy

$ sha256sum -c SHA256SUMS 2>&1 | grep OK

Copy

The sha256sum line should output a line such as:

ame.iso: OK

If the OK for your file appears, that indicates the hash matches.

2. SYSTEM INSTALLATION

👷🛠️UNDER CONSTRUCTION🚧🏗

2.01 Hardware

BIOS Simulators

Hardware Compatibility

Open Hardware

Hardware Security Based

Hardware Vulnerabilities

Hardware Spoof

Secure Boot

Visit our repo tree: 02_SECURITY/02_Systems_Security_Management/03_Secure_Boot
Libreboot
OSResearch

Sanitize options

Visit our repo tree: 02_SECURITY/04_Backup_and_Sanitization/02_Data_Sanitization

Not all SSDs support sanitize. If you use SSDs, enable TRIM in your BIOS. Take care with SSD over-provisioning and SSD flaws. Ensuring TRIM is enabled helps maintain the SSD's performance and longevity by allowing the operating system to inform the SSD which blocks of data are no longer in use and can be wiped internally. Over-provisioning increases the available space for the SSD controller to manage and distribute writes, enhancing performance and durability.

Volatile Memory

Performance

SSD Over-provisioning

This prevents degradation of SSD speed and durability.

Depending on use, some manufacturers recommend 40% OP. For general use, 20% of the general capacity of the SSD, which is about 14% counting the minimum that comes from the factory. For example, a 240GB SSD is limited to -14%, which results in 206GB (34GB of OP).

You must consider the file system you will use.

Cost Benefit

When evaluating cost benefit, consider the total cost of ownership (TCO) which includes purchase price, maintenance, and potential energy savings. Opt for hardware with a good balance between price and durability. Investing in higher quality components may have higher initial costs but often results in lower long-term expenses due to reduced failure rates and longer lifespans.

2.02 Installation

Visit our repo tree: 01_INITIAL_SETUP/01_Installation_Guide

• Basic tips about Linux System
Debian GNU/Linux Installation Guide - https://debian.org/releases/stable/amd64/index.en.html
Stable Release - https://debian.org/releases/stable/
Debian Wiki - https://wiki.debian.org/FrontPage
DontBreakDebian - https://wiki.debian.org/DontBreakDebian
Command Line - https://github.com/jlevy/the-art-of-command-line#everyday-use
Terminal - https://github.com/onceupon/Bash-Oneliner#terminal-tricks
• Bash autocompletion with TAB

$ sudo apt install bash-completion

Copy

Overview of Terminal's Shortcuts

Generally, it's not possible to select and copy text in the terminal using just the keyboard. However, most terminal emulators follow a similar pattern: use SHIFT with the arrow keys to select text, and CTRL + SHIFT + C or CTRL + SHIFT + V to copy and paste, respectively. If these shortcuts aren't working, consider using a terminal multiplexer like screen.

Shortcut	Action	Shortcut	Action
`CTRL + A`	Go to the beginning of the line	`CTRL + E`	Go to the end of the line
`CTRL + F`	Go forward one character	`CTRL + B`	Go back one character
`CTRL + U`	Delete the entire line before the cursor	`CTRL + K`	Delete the entire line after the cursor
`CTRL + W`	Delete the word before the cursor	`CTRL + Y`	Retrieve the last word deleted or cut
`CTRL + T`	Swap the two characters before the cursor	`CTRL + R`	Search command history
`CTRL + C`	Cancel the running command	`CTRL + Z`	Suspends the running command
`CTRL + L`	Clear terminal screen	`CTRL + X`	List possible filename completions
`CTRL + XX`	Toggle between current cursor position and start or end of line	`ESC + T`	Swap the two words before the cursor
`ALT + U`	Capitalize all letters in the word after the cursor	`ALT + L`	Lower case all letters in the word after the cursor
`ALT + .`	Use the last word of the last command	`CTRL + D`	Close the terminal

Overview of Nano's Shortcuts

Basically, use CTRL + O and ENTER to save changes and then CTRL + X to exit the Nano editor.

Shortcut	Action	Shortcut	Action
File Handling		Moving Around
`CTRL + S`	Save current file	`CTRL + B`	One character backward
`CTRL + O`	Offer to write file ("Save as")	`CTRL + F`	One character forward
`CTRL + R`	Insert a file into the current one	`CTRL + ←`	One word backward
`CTRL + X`	Close buffer, exit from Nano	`CTRL + →`	One word forward
		`CTRL + A`	To start of line
Editing		`CTRL + E`	To end of line
`CTRL + K`	Cut current line into cutbuffer	`CTRL + P`	One line up
`ALT + 6`	Copy current line into cutbuffer	`CTRL + N`	One line down
`CTRL + U`	Paste contents of cutbuffer	`CTRL + ↑`	To previous block
`ALT + T`	Cut until end of buffer	`CTRL + ↓`	To next block
`CTRL + ]`	Complete current word	`CTRL + Y`	One page up
`ALT + 3`	Comment/uncomment line/region	`CTRL + V`	One page down
`ALT + U`	Undo last action	`ALT + \`	Top of buffer
`ALT + E`	Redo last undone action	`ALT + /`	End of buffer
Search and Replace		Special Movement
`CTRL + Q`	Start backward search	`ALT + G`	Go to specified line
`CTRL + W`	Start forward search	`ALT + ]`	Go to complementary bracket
`ALT + Q`	Find next occurrence backward	`ALT + ↑`	Scroll viewport up
`ALT + W`	Find next occurrence forward	`ALT + ↓`	Scroll viewport down
`ALT + R`	Start a replacing session	`ALT + <`	Switch to preceding buffer
		`ALT + >`	Switch to succeeding buffer
Deletion		Information
`CTRL + H`	Delete character before cursor	`CTRL + C`	Report cursor position
`CTRL + D`	Delete character under cursor	`ALT + D`	Report line/word/character count
`ALT + BACKSPACE`	Delete word to the left	`CTRL + G`	Display help text
`CTRL + DEL`	Delete word to the right
`ALT + DEL`	Delete current line
Operations		Various
`CTRL + T`	Execute some command	`ALT + A`	Turn the mark on/off
`CTRL + J`	Justify paragraph or region	`TAB`	Indent
`ALT + J`	Justify entire buffer	`SHIFT + TAB`	Unindent marked region
`ALT + B`	Run a syntax check	`ALT + V`	Enter next keystroke verbatim
`ALT + F`	Run a formatter/fixer/arranger	`ALT + N`	Turn line numbers on/off
`ALT + :`	Start/stop recording of macro	`ALT + P`	Turn visible whitespace on/off
`ALT + ;`	Replay macro	`ALT + X`	Hide or unhide the help lines
		`CTRL + L`	Refresh the screen

Virtual Machines

• Using VirtualBox as training or to test ultra-advanced configurations

https://debian.org/doc/manuals/debian-handbook/sect.virtualization.en.html

VirtualBox on Windows 10

"How to Install Debian Linux in VirtualBox on Windows 10 | Beginners Guide | (Buster)"
https://youtube.com/watch?v=cx8GzudB6uE

Hypervisor

KVM, Kernel-based Virtual Machine, is a hypervisor built into the Linux kernel. It is similar to Xen in purpose but much simpler to get running. Unlike native QEMU, which uses emulation, KVM is a special operating mode of QEMU that uses CPU extensions (HVM) for virtualization via a kernel module.

The difference between a type 1 hypervisor and a type 2 hypervisor. KVM is a type 1 hypervisor, it is able to run on bare metal, while QEMU is a type 2 hypervisor, it runs on top of the operating system. QEMU will utilize KVM in order to utilize the machine’s physical resources for the virtual machines. In brief, QEMU uses emulation; KVM uses processor extensions (HVM) for virtualization.

Using QEMU/KVM - "Kernel-based Virtual Machine"

https://wiki.debian.org/KVM
https://wiki.archlinux.org/title/KVM

Visit our repo tree: 06_SYSADMIN/03_Server_Infrastructure/04_Virtualization_Technologies

Quick Installation Guide and Others

https://rodsbooks.com/linux-uefi
https://wiki.debian.org/DontBreakDebian
https://distrowatch.com/table.php?distribution=debian
https://debian.org/releases/bookworm/amd64/apb.en.html

2.2.2 Desktop interface

XFCE vs LXQt - Lightweight Linux Desktop Environments
https://youtube.com/watch?v=cs8JW3zDDoI

2.2.3 Debootstrap

👷🛠️UNDER CONSTRUCTION🚧🏗

Visit our repo tree: 01_INITIAL_SETUP/04_Custom_System/03_Debootstrap

• Debootstrap
https://wiki.debian.org/Debootstrap
• Debian 11.0 Debootstrap | Debian Command Line Install Guide
https://youtube.com/watch?v=oKnkOwdysNs
• Debian 11.4 ZFS Bootstrap | Debian ZFS Command Line Installation
https://youtube.com/watch?v=7F7Ch-ZkiQU
• Nilsmeyer - An ansible role for bootstrapping new Debian based systems, including setting up partitions, file systems, encryption (luks), RAID and LVM
https://github.com/nilsmeyer/ansible-debootstrap
• Linux Dabbler - Scripts to run after installing debian
https://github.com/linuxdabbler/debian-install-scripts

File system: EXT4, XFS, BTRFS AND ZFS

2.03 Encryption

Visit our repo tree: 02_SECURITY/03_Cryptography_and_Passwords

Visit our repo tree: 01_INITIAL_SETUP/04_Custom_System/03_Debootstrap

Encryption

Partitioning scenarios: advantages and disadvantages

FSTAB, CRYPTTAB and DM-CRYPT - Linux kernel's device mapper crypto target

Dm-crypt
Fstab
Crypttab
Tips: Copy and paste blkid to fstab
# echo "$(blkid -o export /dev/sdbX | grep ^UUID=) REMEMBEREFI" | tee --append /etc/fstab
# blkid -o value -s UUID | sudo tee -a /etc/fstab

Example 1 - FSTAB - Non-encrypted Boot Removable Medium Multi-device Compatible


  # / was on /dev/sda1 during installation
  UUID=e4c627c2-69f2-11ee-8c99-0242ac120002 / ext4 errors=remount-ro 0 1
  
  # /boot was on /dev/sdc2 during installation
  PARTUUID=f2c4ec78-69f2-11ee-8c99-0242ac120002 /boot ext2 noauto,x-systemd.device-timeout=1m,defaults 0 2
  
  # /boot/efi was on /dev/sdc1 during installation
  PARTUUID=a15355f4-15ce-4ea6-a57b-161e9eea19fc /boot/efi vfat noauto,x-systemd.device-timeout=1m,umask=0077 0 1
  
  UUID=2701e126-69f3-11ee-8c99-0242ac120002 /home ext4 defaults 0 2
  UUID=447e4a14-69f3-11ee-8c99-0242ac120002 none swap sw 0 0

👷🛠️UNDER CONSTRUCTION🚧🏗

Example 3 - FSTAB - Encrypted Boot Removable Medium (USB Key) Multi-device and Keyfile

Time-based Autentication (OTP, TOTP, and HOTP)

Term	Definition	Example
OTP	One-Time Password	A unique password that is valid for only one login session or transaction.
TOTP	Time-Based One-Time Password	An OTP that is valid for a certain period of time, typically 30 seconds.
HOTP	HMAC-Based One-Time Password	An OTP that is generated based on a counter and a shared secret key.

Nuke Password

$ sudo apt install cryptsetup-nuke-password

Copy

2.04 After Installing

For Everyday Use, Basic Things to do After Installing Debian

1. Adding User to Sudo

How to securely configure sudo for everyday use on your personal Linux machine

The su - command (short for "substitute user") is used to switch to the superuser account, effectively starting a new shell session as the root user. It's the traditional command for switching to the superuser account.

The sudo command allows users to execute specific commands with superuser privileges without switching to the root account. Ultimately, sudo is often preferred for its security benefits and the ability to control and log individual commands, making it suitable for most day-to-day administrative tasks. Some advantages:

Granular Control: sudo enables fine-grained control over which commands a user can run with elevated privileges, reducing the risk of accidental changes.
Audit and Accountability: sudo logs all commands executed, providing a clear audit trail and accountability for administrative actions.
Security: sudo sessions require re-authentication after a period of inactivity, minimizing the risk of unauthorized access if a terminal is left unattended.

Open a Terminal Window:
- Press Ctrl + Alt + T or search for "Terminal" in your application menu.
Switch to the Root User:
Run visudo:
- As the root user, execute:
- ```
# visudo
```
  Copy
- This command opens the sudoers file in your default text editor and ensures that any syntax errors won't be saved, preventing potential misconfigurations.
Add Your User to the Sudoers File:
- Locate the line that includes root ALL=(ALL:ALL) ALL.
- Below that line, add the following line, replacing your_username with your actual username:
- ```
$ your_username ALL=(ALL:ALL) ALL
```
- Save the file and exit the editor:
  - For nano, use Ctrl + O to save and press Enter, press Ctrl + X to exit.

Test Sudo Access

Open a New Terminal Window or Log Out and Log Back In:
- This ensures that the changes take effect.
Test Sudo Access:
- Run a command with sudo to verify:
- ```
$ sudo ls
```
  Copy
- Enter your password when prompted.
- If the command executes successfully, sudo is configured correctly for your user.

Optional: Configure Sudo Timeout

Open the Sudoers Configuration for Timeout:
- Create or edit a specific sudoers configuration file for the timeout setting:
- ```
$ sudo nano /etc/sudoers.d/timeout
```
  Copy
Set the Timeout Value:
- Add or modify the following line to set the timeout (e.g., 3 minutes):
- ```
Defaults timestamp_timeout=3
```
  Copy
- Save the file and exit nano.
- Logoff to take effect.

Additional Tips

Use Sudo Only When Necessary:
- Only use sudo for commands that require elevated privileges to minimize the risk of accidental system changes.
Keep Your System Updated Regularly:
- Run updates frequently to ensure you have the latest security patches and software versions.
Secure Your Password:
- Use a strong and unique password for your user account to enhance security.

Su and Sudo Security Considerations

A strong sudo password should be at least 20 characters long with high entropy, ideally reaching 80 bits. A strong sudo password helps against brute force attacks but does not fully protect against physical acess and malwares.

Restrict sudo access to specific commands using the sudoers file to follow the principle of least privilege. This minimizes the potential damage malware can cause by restricting what it can access even if it compromises a user account.

Avoid having your password stored in the bash history by typing it directly into the command line or in plain text files. Use a secure method to input it or multi-factor authentication (MFA) for additional security.

Physical access: if an attacker has physical access to your machine, they can potentially bypass your sudo measures. Implementing full disk encryption with a strong passphrase is crucial.

Evil Maid Attack: physical access could also allow an attacker to modify boot parameters or boot from an external device. Secure boot configurations and BIOS/UEFI passwords can help mitigate these risks.

Malwares or phishing attacks: They can undermine both sudo and su by capturing passwords or escalating privileges. Regularly update your system and use antivirus tools to detect and prevent malware and verify urls. Regularly update your system and applications to patch security vulnerabilities. Employing measures such as Intrusion Detection Systems (IDS) and url checking can also help monitor and alert you to suspicious activities.

Alternatives to sudo

suexec: Often used in web server environments, this tool allows a user to execute commands as another user, particularly for running CGI scripts as a non-root user to enhance security.

Consider using doas as an alternative to sudo for simpler configuration and usage.

Consider using gksu or pkexec for graphical applications requiring elevated permissions.

2. Connect to WiFi

UNDER CONSTRUCTION

3. Update and Upgrade

https://itsfoss.com/apt-upgrade-vs-dist-upgrade

$ sudo apt update -y && sudo apt upgrade -y

Copy

You can modify `sources.list` if you wish to change the list of mirrors from CD-ROM

https://wiki.debian.org/SourcesList
https://backports.debian.org/Instructions

Example to overwrite sources.list with Debian Bookworm backports sources.


  $ echo '
  
  deb https://deb.debian.org/debian/ bookworm main contrib non-free-firmware
  deb-src https://deb.debian.org/debian/ bookworm main contrib non-free-firmware
  
  deb https://deb.debian.org/debian/ bookworm-updates main contrib non-free-firmware
  deb-src https://deb.debian.org/debian/ bookworm-updates main contrib non-free-firmware
  
  deb https://security.debian.org/debian-security/ bookworm-security main contrib non-free-firmware
  deb-src https://security.debian.org/debian-security/ bookworm-security main contrib non-free-firmware
  
  deb https://deb.debian.org/debian bookworm-backports main contrib non-free
  deb-src https://deb.debian.org/debian bookworm-backports main contrib non-free
  
  ' | sudo tee /etc/apt/sources.list

Notes about system update commands

Command	Description
`sudo apt update`¹	Fetches the package lists from the repositories and updates them to get information on the newest versions of packages and their dependencies. This command does not install or upgrade any packages, but it's a necessary first step before any upgrade operation.
`sudo apt upgrade`²	Upgrades all the packages that have newer versions available, according to the package lists fetched by `apt update` . This command upgrades the installed packages without removing any currently installed packages or installing new ones that are not already installed.
`sudo apt dist-upgrade`³	In addition to performing the function of `apt upgrade` , this command handles changing dependencies with new versions of packages. It might remove some packages if necessary to complete the upgrade, ensuring the system remains in a consistent state.
`sudo apt full-upgrade`⁴	Similar to `apt dist-upgrade` , this command upgrades the system by installing new package versions and handling dependency changes, including package removals if necessary. The `full-upgrade` command is synonymous with `dist-upgrade` in `apt` .

References

Debian Manual Page for apt(8) . Available at: https://manpages.debian.org/buster/apt/apt.8.en.html
Debian Reference - Package Management. Available at: https://debian.org/doc/manuals/debian-reference/ch02.en.html
APT User Manual. Available at: https://manpages.debian.org/buster/apt/apt.8.en.html

4. Firewall

Install and enable Uncomplicated Firewall - UFW (CLI)

$ sudo apt install ufw

$ sudo ufw enable

$ sudo ufw default deny incoming

$ sudo ufw default allow outgoing

$ sudo ufw status verbose

5. SSH

UNDER CONSTRUCTION
https://keepassxc.org/docs/KeePassXC_UserGuide#_openssh_agent_on_linux

6. Installing Java

It's necessary if you install e.g. Zotero or JDownloader2.

• Java Runtime Environment (JRE)
• OpenJDK - Java Development Kit (JDK)

$ java --version

$ apt-cache search openjdk | grep openjdk

$ apt search openjdk

$ sudo apt install default-jre

7. Installing Micro$oft Fonts

This is necessary if you need certain fonts for academic formatting. Microsoft fonts, such as Times New Roman, Arial, and Courier New, are required in some common academic formatting styles, including APA, MLA, Chicago, and IEEE.

$ sudo apt install -y ttf-mscorefonts-installer

Copy

Open source alternatives to these fonts are available:

Times New Roman:
- Liberation Serif: Designed to be metrically compatible with Times New Roman.
- Nimbus Roman: A serif typeface that is very similar to Times New Roman.
- DejaVu Serif: Another good open-source serif font with broad language support.
Arial:
- Liberation Sans: Designed to be metrically compatible with Arial.
- Nimbus Sans: A sans-serif typeface similar to Arial.
- DejaVu Sans: An open-source sans-serif font with extensive language support.
Courier New:
- Liberation Mono: Designed to be metrically compatible with Courier New.
- Nimbus Mono: A monospaced typeface similar to Courier New.
- DejaVu Sans Mono: A good open-source monospaced font with broad language support.

Nerd Fonts

Nerd Fonts is a project that patches developer targeted fonts with a high number of glyphs (icons). Specifically to add a high number of extra glyphs from popular 'iconic fonts' such as Font Awesome, Devicons, Octicons, and others.

https://github.com/ryanoasis/nerd-fonts

8. Fast Grub Time

Edit Grub timeout

$ sudo nano /etc/default/grub

Copy

GRUB_TIMEOUT=0

Copy

$ sudo update-grub

Copy

9. Swapiness

https://unix.stackexchange.com/questions/265713/how-to-configure-swappiness-in-linux-memory-management

Verify

$ sudo cat /proc/sys/vm/swappiness

Copy

Edit and add sw.swappiness=10

$ sudo nano /etc/sysctl.conf

Copy

sw.swappiness=10

Copy

or simply

$ echo 'vm.swappiness = 10' | sudo tee -a /etc/sysctl.conf

Copy

To take effect:

$ sudo sysctl -p

Copy

sysctl vm.swappiness=10

Copy

Verify

$ sudo cat /proc/sys/vm/swappiness

Copy

*Note: for security reasons, use encrypted swap partiton.

10. TLP

Do not install TLP (Optimize Linux Laptop Battery Life, Link ), avoid problems.

2.05 OS Deployment

Ventoy - A New Bootable USB Solution

Ventoy enables creating a versatile bootable USB drive capable of booting multiple ISO files directly without the need for reformatting each time.

Download Ventoy
Ventoy Releases on GitHub - Source code and latest releases

Getting Started with Ventoy

Ventoy Documentation - Getting Started - Installation guide and basic usage
How to Use Ventoy for Multi-ISO Boot on USB Flash Drive - Guide for using multiple ISOs on a single USB drive
Ventoy Tutorial: Create Bootable USB Drive for Windows 11/10 - Step-by-step tutorial for Windows users

Secure Boot in UEFI Mode

Ventoy Documentation - Secure Boot in UEFI Mode - Information on configuring Ventoy with Secure Boot

Ventoy Persistence Plugin

Ventoy Persistence Plugin Documentation - Guide to using persistence for rescue disk purposes

(*recommended only for rescue disk purposes)

2.06 Disk Cloning

Using dd Command for Disk Cloning

Using Ventoy with Debian Netinst or Alpine Linux for disk cloning.

The dd is a versatile command-line tool for low-level copying and conversion of data, capable of cloning entire disks or partitions directly. It operates at a lower level, it gets raw format and the drive is copied to a file using bit-by-bit method, requiring precise device specifications to avoid data loss.

In contrast,Clonezilla offers a user-friendly interface for disk imaging and cloning tasks. It supports a wider range of file systems, allows selective partition cloning, and includes features like disk resizing and encryption. It provides more convenience and options compared to dd, making it suitable for users preferring simplicity and additional functionalities.

1. First Steps

Visit the Ventoy (Download) page and download the latest release suitable for your operating system. Install Ventoy following the instructions provided.

Boot your computer from the Ventoy-prepared USB drive. Achieve a shell command line to proceed with disk cloning.

1. Boot Menu Selection: After booting from the Ventoy USB drive, you will see a boot menu displaying all the bootable ISO images present on the drive.
2. Select the Tool or Shell: Navigate through the Ventoy boot menu to find and select the tool or shell environment that provides a command line interface. This could be labeled as "Shell," "Command Line," "Advanced Options," or similar.
3. Accessing Teletype or Shell: Once you select the appropriate option, the system will boot into a teletype (TTY) or shell environment. This environment gives you direct access to a command line interface where you can execute commands like dd for disk cloning.

2. Using `dd` Command for Disk Cloning

A) For Debian Netinst:

Debian typically includes necessary utilities like dd, md5sum, and gzip by default. You can proceed with the following commands:

Identify Source and Destination Disks

Use tools like lsblk or fdisk -l to identify the device names of both the source (input) and destination (output) disks.

Execute the Command with Compression (Optional)

To clone a disk using dd and optionally compress the output with gzip, use:

$ sudo dd if=/dev/sda | gzip >
  /path/to/output/image.gz

if=/dev/sda: Specifies the input file (source disk).
| gzip > /path/to/output/image.gz: Pipes the output of dd to gzip for compression and saves it as image.gz.

Adjust paths and filenames according to your specific setup.

Compute MD5 Hash of Compressed Image to check integrity of the image (Optional)

To compute the MD5 hash of the compressed image file (image.gz), use:

$ md5sum /path/to/output/image.gz

This command calculates the MD5 checksum of image.gz. The output will be a 32-character hexadecimal number, which is the MD5 hash.

Compare the computed MD5 hash with the original hash to verify the integrity:

$ md5sum -c checksum.txt

Ensure checksum.txt contains the MD5 hash along with the filename, like:

9f16e3f68c548804c0dca6cfb0cfc6a2
  /path/to/output/image.gz

If both hashes match, the file integrity is confirmed.

For more information: md5sum command manual page.

B) For Alpine Linux:

Alpine Linux is a minimalist distribution and may require installing necessary utilities. Use the following commands to install dd, md5sum, and gzip if they are not already available:


  # Update package repositories
  $ apk update
  # Install required utilities
  $ apk add coreutils gzip

After installing the necessary utilities, proceed with the disk cloning and verification steps similar to Debian.

Clonezilla

Clonezilla is a powerful, free, and open-source software designed for disk imaging and cloning purposes.

Clonezilla Documentation

How to Use Clonezilla Software

WikiHow Guide: How to Use Clonezilla

Clonezilla offers several methods for disk imaging and cloning, primarily focused on local and network-based operations:

Disk Image:
- Integral: Creates an exact copy of the entire disk, including all partitions and data.
- Partial: Allows selective cloning of specific partitions on a disk.
Disk-to-Disk: Clones one disk directly to another, including all partitions and data.
Partition-to-Partition: Clones specific partitions from one disk to another.
Partition-to-Image: Saves selected partitions as an image file, which can be stored on another disk or network location.
Image-to-Partition: Restores partitions from a saved image file back onto a disk.
Image-to-Disk: Restores an entire disk from a saved image file.

Clonezilla primarily operates offline, meaning it does not clone disks or partitions over the internet directly. It requires direct access to the disks or uses network-based filesystems like NFS or Samba for remote operations.

Changing disk name

Command

$ cnvt-ocs-dev -d /home/partimag 'image' 'sda3' 'sda2'

2.07 Low Level Linux

Kernel Linux
https://kernel.org
Linux Training
https://training.linuxfoundation.org
A Beginner's Guide to Linux Kernel Development (LFD103)
Linux From Scratch (LFS)
https://linuxfromscratch.org
Reproducible Builds
https://reproducible-builds.org
Unix & Linux Stack Exchange
https://unix.stackexchange.com
Linux Journal
https://linuxjournal.com
Linux Performance
https://brendangregg.com/linuxperf.html
Linux Assembly Language Programming
https://asmtutor.com

3. SECURE BOOT

👷🛠️UNDER CONSTRUCTION🚧🏗

3.01 Introduction

"Most modern systems will ship with SB enabled - they will not run any unsigned code by default, but it is possible to change the firmware configuration to either disable SB or to enroll extra signing keys. The whole point of Secure Boot is to prevent malware from gaining control of the computer. Therefore, when booting with Secure Boot active, Fedora 18 and later, Ubuntu 16.04 and later, and probably other distributions restrict actions that some Linux users take for granted. For instance, Linux kernel modules must be signed, which complicates use of third-party kernel drivers, such as Nvidia's and AMD/ATI's proprietary video drivers. More recent kernels may, if Secure Boot is active, also check that they were launched from a boot loader that honors Secure Boot, and shut down if this was not the case."

"To launch a locally-compiled kernel, you must sign it with a MOK and register that MOK with the system. (In both cases, you can register a hash rather than sign the binary; but this approach results in an ever-growing database in NVRAM, which is undesirable.) The extent of such restrictions is entirely up to those who develop and sign the boot loader launched by Shim and the kernel launched by that boot loader, though. Some distributions ship kernels that are relatively unencumbered by added security restrictions."

"As a practical matter, if you want to use Shim, you have two choices: You can run a distribution that provides its own signed version of Shim, such as Fedora 18 or later or Ubuntu 12.10 or later; or you can run a signed version from such a distribution or from another source, add your own MOK, and sign whatever binaries you like. This first option is quite straightforward if you happen to want to use a distribution that ships with Shim, and it requires little extra elaboration.If you want to build and run your own kernel (e.g. for development or debugging), then you will obviously end up making binaries that are not signed with the Debian key. If you wish to use those binaries, you will need to either sign them yourself and enroll the key used with MOK or disable SB."

  CAUTION:
  • Use an administrator password in the BIOS and do not use the same
  for disk encryption.
  • Building and signing kernel modules is independent of building and
  signing your own kernel.
  • In Debian, if you do not install the DKMS package, you will have
  more work to create the X.509keys or OpenSSL keys, import the keys
  with sbsigntool or mokutil, sign the kernel or the kernelmodule file
  with sbsigntool or sign-file, respectively.
  • Debian comes with signed kernels to work with your GRUB so it will
  most likely not benecessary to sign the kernel that includes Debian,
  however any foreign kernel or compiled fromits source kernel.org must
  be signed or will not be able to load.
  • Ubuntu uses DKMS with signed key by default, Ubuntu creates and
  imports mok key during systeminstallation.
  • In Fedora, if you use DKMS with Secure Boot enabled, you have to
  import the DKMS sign keywith mokutil --import /var/lib/dkms/mok.pub
  and reboot to enroll the key. In Fedora the mok.puband mok.key keys
  are created and module is signed by DKMS, but only if openssl
  packageis installed.
  • UEFI specifications use the terms key and public key (.der) to mean
  the public part of thekey pair, or the X.509 certificate. However, in
  OpenSSL, the term key is the private key (.priv)that's used for
  signing. Thus, all Secure Boot keys must be X.509 keys and not OpenSSL
  keys.
  • The instructions provided assume that you're signing a module
  for the currently runningkernel. If you're signing a module for a
  different kernel, you must provide the path to thesign-file utility
  within the correct kernel version source. Otherwise, the signature
  typefor the module for that kernel might not align correctly with the
  expected signature type.
  • Only a single custom certificate can be added to the kernel because
  the compressed sizeof the kernel's boot image can ot increase. Do
  not add multiple certificates to the kernelboot image.

  WARNING:
  https://makedebianfunagainandlearnhowtodoothercoolstufftoo.computer/doku.php?id=start:issecurebootworking
  https://discourse.ubuntu.com/t/dkms-package-support-extra-drivers-does-not-work-in-ubuntu-22-10-install-media/31655

  BUGS:
  • Debian Bug report logs - #1037146 Key was rejected by service -
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1037146 -
  • Debian Bug report logs - #1012741 Key was rejected by service -
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012741 -
  • Debian Bug report logs - #1012816 Key was rejected by service -
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1012816;msg=22 -
  • Debian Bug report logs - #989463 please align shim-signed dkms
  behaviour with Ubuntu -
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989463 -
  • Debian Bug report logs - #939392 please provide kmodsign like Ubuntu
  does - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939392 -
  • Debian Bug report logs - #928300 shim-signed: secure boot via
  removable media path unavailable -
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=928300

3.02 Secure Boot References

3.03 YouTube References

3.04 Sign GRUB

https://wiki.archlinux.org/title/Kernel_parameters

Debian comes with signed kernels to work with your GRUB so it will most likely not be necessary to sign the kernel that includes Debian, however any foreign kernel or compiled from its source kernel.org must be signed or will not be able to load.

3.05 Sign Debian Kernel

Debian comes with signed kernels to work with your GRUB so it will most likely not be necessary to sign the kernel that includes Debian, however any foreign kernel or compiled from its source kernel.org must be signed or will not be able to load.

Only a single custom certificate can be added to the kernel because the compressed size of the kernel's boot image can not increase. Do not add multiple certificates to the kernel boot image.

1.First steps

All the items below have to do with SecureBoot mode.


  $ sudo mokutil --sb-state SecureBoot enabled

If controlling the Secure Boot state through the EFI setup program is difficult, you can optionally use the mokutil utility to disable Secure Boot at the level of the Shim so that, although UEFI Secure Boot is enabled, o further validation takes place after the Shim is loaded.

What keys are on my system?

user@debian:~$ sudo mokutil --list-enrolled

or


  $ sudo mokutil --list-enrolled | grep Subject:

Also the command modinfo prints the signature if available, for example:


  $ sudo modinfo
  /lib/modules/6.1.0-11-amd64/kernel/mm/zsmalloc.ko

2.Place to auto-generated MOK

MOK - Machine Owner Key

Introduction

The use of mokutil that's most relevant to this page is to import a MOK. In this context, importing refers to storing a MOK in the computer's NVRAM, along with a flag to tell Shim and MokUtil that the MOK is there and ready to be enlisted when you next reboot the computer. Keys can be added and removed in the MOK list by the user, entirely separate from the distro CA key. Unlike Debian, Ubuntu has chosen to place their auto-generated MOK at "/var/lib/shim-signed/mok/", which some software--such as Oracle's virtualbox package -expect to be present. Note that using this same location may result in future conflicts. Warning: The MOK.key file is extremely sensitive! An attacker who gains access to it could generate binaries that your computer would accept as authorized. You should change permissions to prevent unauthorized access, and ideally store it on an encrypted external storage medium and unplug it when you're not signing binaries.If you see the key there (consisting of the files MOK.der, MOK.pem and MOK.priv) then you can use these, rather than creating your own.

First make sure the key doesn't exist yet:


  $ ls /var/lib/shim-signed/mok/

To create a folder to MOK key:


  $ sudo mkdir -p /var/lib/shim-signed/mok/

You can choose another placcautione like "/etc/mok_key/" since there is no standard location at the moment.


  $ sudo mkdir -p /etc/mok_key/

3.Generating a new key

Before you create the public and private key for signing the kernel, you eed to access the folder you created to be the destination of the keys. Then create the public (mokcertificate.der) and private key (moksigningkey.priv) with one-time password for signing the kernel


  $ cd /var/lib/shim-signed/mok/
  $ sudo openssl req -config
  $(openssl version -d) -new -x509 -newkey rsa:2048 -keyout MOK.priv
  -outform DER -out MOK.der -days 36500 -subj
  "/CN=ShimSigned/"


  $ sudo openssl x509 -in MOK.der -inform DER -outform PEM -out MOK.pem
  $ ls -l total 12 -rw-r--r-- 1 root root787MOK.der -rw-r--r-- 1 root
  root 1123MOK.pem -rw------- 1 root root 1854MOK.priv
  $ sudo chmod 600 /var/lib/shim-signed/mok/*

This commands will create both the private and public part of the certificate to sign things. You need both files to sign; and just the public part (MOK.der) to enroll the key in Shim.

To read the certificate file in a human readable format, use


  $ sudo openssl x509 -in /var/lib/shim-signed/mok/MOK.pem -noout -text

Another example of key generation:


  $ sudo openssl req -x509 -new -nodes -utf8 -sha512 -days 3650 -batch
  -config /etc/ssl/x509.conf -outform DER -out /etc/ssl/certs/pubkey.der
  -keyout /etc/ssl/certs/priv.key
  $ sudo openssl x509 -inform DER -in /etc/ssl/certs/pubkey.der -out
  /etc/ssl/certs/pubkey.pem

4.Enrolling your key im Shim

Enroll the key to your installation:


  $ cd /var/lib/shim-signed/mok/ 
  $ sudo mokutil --import MOK.der

You will be asked for a one-time password (remember it and type it correctly) , you will just use it to confirm your key selection in the next step (you won't need this password beyond this point, though), so choose any.

Recheck your key will be prompted on next boot


  $ sudo mokutil --list-new

5.Restart and finsh the process

Restart your system. Changes to the MOK keys may only be confirmed directly from the bash at boot time. You will encounter a blue screen of a tool called MOKManager. Select " Enroll MOK" and then "View key". Make sure it is your key you created in step 3. Afterwards continue the process and you must enter the password which you provided in step 4. Continue with booting your system.

Verify your key is already enrolled, if the MOK was loaded correctly, with:


  $ sudo mokutil --test-key
  /var/lib/shim-signed/mok/MOK.der

6.Sign your installed kernel or modules

6.1 Modern Method: Signing the Debian kernel and modules with DKMS

Building Debian kernel modules with DKMS. The dkms frameworks allows building kernel modules "on the fly" on the local system instead of building them centrally on the Debian infrastructure, DKMS could automatically sign kernel updated modules. If you install the kernel modules through the apt repository, chances are that modules have already been signed by the DKMS signing key. In that case, the traditional method won't work. And the thing you only need to do is to enroll the DKMS signing key into your machine. On systems that use SecureBoot, you will need a Machine Owner Key (MOK) to load DKMS modules. Generate it, enroll it, sign modules with it and then you will be able to load the signed modules.

In Debian, it depends on the dkms package:


  $ sudo apt install dkms

In order for dkms to automatically sign kernel modules, it must be told which key to sign the module with. This is done by adding two configuration values to "/etc/dkms/framework.conf", adjusting paths as required:

mok_signing_key="/var/lib/shim-signed/mok/MOK.priv"

mok_certificate="/var/lib/shim-signed/mok/MOK.der"

DKMS Sign Helper Script

If these values are provided and dkms is able to build modules but does ot attempt to sign them, then it is likely that sign_tool is missing. This is more common in older and/or custom kernels. In "/etc/dkms/framework.conf", add:

sign_tool="/etc/dkms/sign_helper.sh"

Create "/etc/dkms/sign_helper.sh" with:


  /lib/modules/"$1"/build/scripts/sign-file sha512
  /root/.mok/client.priv /root/.mok/client.der "$2"

Set Linux kernel info variables


  $ VERSION="$(uname -r)"
  $ SHORT_VERSION="$(uname -r | cut -d . -f 1-2)"
  $ MODULES_DIR=/lib/modules/$VERSION
  $ KBUILD_DIR=/usr/lib/linux-kbuild-$SHORT_VERSION

Making DKMS modules signing by DKMS signing key usable with the secure boot

If you install the kernel modules through the apt repository, chances are that modules have already been signed by the DKMS signing key. In that case, the traditional method won't work. And the thing you only need to do is to enroll the DKMS signing key into your machine. Here is how we can do that:

First, use the method mentioned in Verifying if a module is signed to check if the modules are signed by DKMS signing key.

Next, find the location of the mok signing key and mok certificate. You can view the location in /etc/dkms/framework.conf, and the default location is /var/lib/dkms.

Then, run the following command to enroll the key into the machine:


  $ sudo mokutil --import /var/lib/dkms/mok.pub # prompts for one-time
  password and /var/lib/mok.pub can be changed, if mok certificate
  isn't located there.
  $ sudo mokutil --list-new # recheck your key will be prompted on ext
  boot !rebooting machine then enters MOK manager EFI utility: enroll
  MOK, continue, confirm, enter password, reboot!
  $ sudo dmesg | grep cert # verify your key is loaded

6.2 Traditional Method: signing the Debian kernel with sbsigntool

Building and signing modules is independent of building and signing your own kernel (vmlinuz). To sign a custom kernel or any other EFI binary you want to have loaded by Shim, you’ll need to use a different command: sbsign (PEM). In this case, we’ll need the certificate in a different format, mokutil nneeds DER, sbsign nneeds PEM.

Convert the certificate into PEM (.der to .pem), for example:

$ sudo openssl x509 -in MOK.der -inform DER -outform PEM -out
  MOK.pem

For example, use it to sign our Kernel:


  $ sudo sbsign --key MOK.priv --cert MOK.pem
  "/boot/vmlinuz-$VERSION" --output
  "/boot/vmlinuz-$VERSION.tmp"
  $ sudo mv "/boot/vmlinuz-$VERSION.tmp"
  "/boot/vmlinuz-$VERSION"

For example, use it to sign our EFI binary:


  $ sudo sbsign --key MOK.priv --cert MOK.pem grubx64.efi --output
  grubx64.efi.signed
  $ sudo mv "grubx64.efi.signed" "grubx64.efi"

Sign the installed Kernel using the key created according to the location you gave it, this will create a new signed vmlinuz. Sign vmlinuz using sbsign and .pem certificate, it should be at /boot/vmlinuz-[KERNEL-VERSION].

To check your Kernel version, you can also use the command:

$ uname -r 6.1.0-12-amd64

Signing vmlinuz (kernel) using sbsign:

$ sudo sbsign
  --key MOK.priv --cert MOK.pem /boot/vmlinuz-[KERNEL-VERSION] --output
  /boot/vmlinuz-[KERNEL-VERSION].signed

For example

$ sudo sbsign --key
  /var/lib/shim-signed/mok/MOK.priv --cert
  /var/lib/shim-signed/mok/MOK.pem
  "/boot/vmlinuz-6.1.0-12-amd64" --output
  "/boot/vmlinuz-6.1.0-12-amd64.signed"

Alternatively:

$ cd /var/lib/shim-signed/mok/
  $ sudo sbsign --key MOK.priv --cert MOK.pem
  "/boot/vmlinuz-[KERNEL-VERSION] --output
  "/boot/vmlinuz-[KERNEL-VERSION].signed"

Remove the unsigned one and restore the original name of the signed one, this will create a new signed vmlinuz:

$ sudo mv
  "/boot/vmlinuz-6.1.0-12-amd64.signed"
  "/boot/vmlinuz-6.1.0-12-amd64"

Update your grub-config

$ sudo update-grub

Reboot your system and select the signed kernel. Now your system should run under a signed kernel and upgrading GRUB2 works again. If you want to upgrade the custom kernel, you can sign the new version easily by following above steps again from step seven on. Thus BACKUP the MOK-keys (MOK.der, MOK.pem, MOK.priv) in an encrypted device.

Verifying if a module is signed. The command modinfo prints the signature if available, for example:

$ sudo modinfo
  /boot/vmlinuz-6.1.0-12-amd64

Others commands


  $ sudo dmesg | grep cert
  $ sudo sbverify --list /boot/vmlinuz-6.1.0-12-amd64
  $ sudo sbverify --cert /etc/mok_key/mok.crt
  /boot/vmlinuz-6.1.0-12-amd64

3.06 Reset Secure Boot Keys

Reset Secure Boot keys for Kernel or Modules

Reset Key for Kernel

👷🛠️UNDER CONSTRUCTION🚧🏗

https://rodsbooks.com/efi-bootloaders/controlling-sb.html#setuputil

"The ASUS permits to you restore the default keys, so this isn't really vital if you're starting from the factory defaults with this model; but if yours doesn't offer such a reset-to-defaults option or if you've modified the keys, saving them may be prudent. As the name implies, this option also erases all your Secure Boot keys. (It does not erase your MOKs, though.)"

Reset MOK Keys for Modules

👷🛠️UNDER CONSTRUCTION🚧🏗

https://rodsbooks.com/efi-bootloaders/controlling-sb.html#key-revocation


  $ sudo mokuitil --sb-state SecureBoot disabled


  $ sudo mokutil --disable-validation

Backup. Exports to list (ideally store it on an encrypted external storage medium).

$ sudo mokutil --export

To remove all (MOKs being a list and not just a single MOK, you can make the shim trust keys from several different vendors, allowing dual and multi-boot)


  $ sudo mokutil --reset --mok


  $ sudo mokutil --reset

To remove one key, first show all keys.


  $ sudo ls -1 MOK*

Shows you keys enrolled.


  $ sudo mokutil --list-enrolled | grep Subject:

Delete those not enrolled to maintain secure boot.


  $ sudo mokutil --delete MOK-0001.der

Uninstall the modules, if it was made with script "make".


  $ cd ~/realtekwifi
  $ sudo make uninstall

or


  $ sudo rmmod 8192eu sudo rmmod rtl8xxxu sudo dkms remove -m rtl8192eu
  -v 1.0 or sudo lshw -C network

Reset de modules and unload them in Kernel


  $ sudo depmod -a -v
  $ sudo update-initramfs -k all -u -v

3.07 OpenSSL Errors

Error 1 - No such file

At main.c:298: - SSL error:FFFFFFFF80000002:system
  library::No such file or directory: ../crypto/bio/bss_file.c:67 - SSL
  error:10000080:BIO routines::no such file:
  ../crypto/bio/bss_file.c:75

Error 2 - Unable to get passphrase

At main.c:170: - SSL error:07880109:common libcrypto
  routines::interrupted or cancelled: ../crypto/passphrase.c:184 - SSL
  error:07880109:common libcrypto routines::interrupted or cancelled:
  ../crypto/passphrase.c:184 - SSL error:1C80009F:Provider
  routines::unable to get passphrase:
  ../providers/implementations/encode_decode/decode_epki2pki.c:96 - SSL
  error:07880109:common libcrypto routines::interrupted or cancelled:
  ../crypto/passphrase.c:184 - SSL error:04800068:PEM routines::bad
  password read: ../crypto/pem/pem_pkey.c:155 sign-file:
  /var/lib/shim-signed/mok/MOK.priv: Success

Possible Causes

Certificate or key are missing. That statement is telling us one of both files that DKMS or OpenSSL.conf are looking for are not where it is looking. Another possibility is that to sign a custom kernel or any other EFI binary you want to have loaded by shim, you’ll need to use a different command: sbsign or mokutil. Unfortunately, we’ll need the certificate in a different format in this case, mokutil needs DER, sbsign needs PEM. Convert the certificate into PEM (.der to .pem).

Under normal conditions, when CONFIG_MODULE_SIG_KEY is unchanged from its default, the kernel build will automatically generate a new keypair using openssl if one does not exist in the file:

certs/signing_key.pem during the building of vmlinux (the
  public part of the key nneeds to be built into vmlinux) using
  parameters in the: certs/x509.genkey file (which is also generated if
  it does not already exist).

It is strongly recommended that you provide your own x509.genkey file.

As long as the signing key is enrolled in shim and does not contain the Object Identifier (OID) from earlier (since that limits the use of the key to kernel module signing), the binary should be loaded just fine by shim.

Cause 1

Wrong syntax of sign-file


  $ sudo scripts/sign-file sha512 kernel-signkey.priv
  kernel-signkey.x509 module.ko

https://kernel.org/doc/html/v4.15/admin-guide/module-signing.html

Cause 2

This is where Debian places openssl.cnf for the OpenSSL they provide:


  $ openssl version -d OPENSSLDIR: "/usr/lib/ssl"
  $ ls -l /usr/lib/ssl lrwxrwxrwx 1 root root mmm 30 mm:mm openssl.cnf
  -> /etc/ssl/openssl.cnf
  $ ls -l /etc/ssl/ -rw-r--r-- 1 root root mmm 30 mm:mm openssl.cnf

It is kind of buried in OpenSSL source code for apps.c, load_config and what happens when openssl.cnf is NULL (i.e., no -config option or OPENSSL_CONF envar). When openssl.cnf is NULL and no overrides, then OPENSSLDIR is used.

Cause 2

Wrong syntax of OpenSSL

Man Page OpenSSL:

Man OpenSSL


  $ sudo openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 -batch
  -config openssl.cnf -outform DER -out MOK.der -keyout MOK.priv
  $ sudo openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 -batch
  -outform DER -out MOK.der -keyout MOK.priv
  $ sudo openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 -batch
  -config openssl.cnf -outform DER -out MOK.der -keyout MOK.priv
  $ sudo openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 -batch
  -outform DER -out MOK.der -keyout MOK.priv

Ubuntu:

https://ubuntu.com/blog/how-to-sign-things-for-secure-boot


  $ sudo openssl req -config ./openssl.cnf -new -x509 -newkey rsa:2048
  -nodes -days 36500 -outform DER -keyout "MOK.priv" -out
  "MOK.der"

Debian:

https://wiki.debian.org/SecureBoot


  $ sudo openssl req -new -x509 -newkey rsa:2048 -keyout MOK.priv
  -outform DER -out MOK.der -days 36500 -subj "/CN=My Name/"
  $ sudo openssl x509 -inform der -in MOK.der -out MOK.pem

Fedora:

https://docs.fedoraproject.org/en-US/quick-docs/kernel-build-custom/


  $ sudo openssl req -new -x509 -newkey rsa:2048 -keyout
  "key.pem" -outform DER -out "cert.der" -nodes
  -days 36500 -subj "/CN= yourname/"

Solutions

Solution 1:


  $ sudo dpkg -S sign-file

Solution 2:

Location


  $ openssl version -d

You can use strace (man strace) to check the configuration file being used while generating the self-signed certificate.


  $ strace -e trace=open,openat -o /tmp/strace.log.0 openssl req \
  -newkey rsa:2048 -x509 -nodes -keyout localhost.key \
  -new -out localhost.crt
  $ grep "openssl.cnf" /tmp/strace.log.0 openat(AT_FDCWD,
  "/etc/pki/tls/openssl.cnf", O_RDONLY) = 3 sudo cat
  /etc/ssl/openssl.cnf openssl_conf = openssl_init from
  /etc/ssl/openssl.cnf

To override system default with user level environment. An empty file will do:

touch ~/.openssl.cnf

BASH define & export:

export OPENSSL_CONF=~/.openssl.cnf

Wrap application within a script:

export OPENSSL_CONF=/dev/null

Solution 3:


  $ openssl version -d

You can use strace (man strace) to check the configuration file being used while generating the self-signed certificate.


  $ strace -e trace=open,openat -o /tmp/strace.log.0 openssl req \
  -newkey rsa:2048 -x509 -nodes -keyout localhost.key \
  -new -out localhost.crt
  $ grep "openssl.cnf" /tmp/strace.log.0 openat(AT_FDCWD,
  "/etc/pki/tls/openssl.cnf", O_RDONLY) = 3 sudo cat
  /etc/ssl/openssl.cnf openssl_conf = openssl_init from
  /etc/ssl/openssl.cnf

To override system default with user level environment. An empty file will do:

touch ~/.openssl.cnf

BASH define & export:

export OPENSSL_CONF=~/.openssl.cnf

Wrap application within a script:

export OPENSSL_CONF=/dev/null

Solution 3:

Rescue if install/build fails in previous step


  $ sudo apt install -f
  $ sudo dpkg-reconfigure broadcom-sta-dkms

3.08 Sign Wi-Fi

How to get Wi-Fi Module signed for Secure Boot

Mandatory packages: openssl, sign-file, and mokutil.

If you are going to compile the module in the kernel, usually the maintainer will indicate the packages to be installed beforehand. For example, you will need to install "make", "gcc", "kernel headers", "kernel build essentials", and "git".

$ sudo apt install git make gcc
  build-essential linux-image-$(uname -r|sed 's,[^-]*-[^-]*-,,')
  linux-headers-$(uname -r|sed 's,[^-]*-[^-]*-,,')

Brief - Sign with Sign-file

  1- Install a driver and test without Secure Boot
  2- Enable Secure Boot
  3- Generate a private and public keys
  5- Import
  6- Reboot and Enroll
  4- Sign the module with sign-file

1. Check if secure boot is enabled.

When Secure Boot is disabled, the shimx64.efi will just directly load the real grubx64.efi bypassing all the Secure Boot steps, including loading the MOK. With the MOK not loaded, the kernel will have no way to recognize the signature on your module as valid. And with Secure Boot disabled, a signed module with an invalid signature is rejected, while unsigned modules only get a warning and a taint mark on any future oops/panic messages.


  $ sudo mokutil --sb-state

2. Generate a personal public/private RSA key pair to sign the kernel modules.

You can chose to store the key/pair, for example, in the /var/lib/shim-signed/mok/ directory. Then create a new pair of private key (MOK.priv) and public key (MOK.der).


  $ sudo mkdir -p /var/lib/shim-signed/mok
  $ sudo openssl req -config /usr/lib/ssl/openssl.cnf -new -x509 -newkey
  rsa:2048 -nodes -days 36500 -outform DER -keyout
  "/var/lib/shim-signed/mok/MOK.riv" -out
  "/var/lib/shim-signed/mok/MOK.der" -subj "/CN=MODULE/"   $ ls -l
  /var/lib/shim-signed/mok/
  total 8
  -rw-r--r-- 1 root root 779 MOK.der
  -rw------- 1 root root 1704 MOK.priv
  $ sudo chmod 600 /var/lib/shim-signed/mok/*

3. Enroll the public key (MOK.der) to MOK (Machine Owner Key).


  $ sudo mokutil --import /var/lib/shim-signed/mok/MOK.der
  input password:
  input password again:

Recheck if your key will be prompted on next boot:


  $ sudo mokutil --list-new

4. Reboot and Enroll

The password in this step is a temporary use password you'll only need to remember for a few minutes. Reboot the machine. When the bootloader starts, you should see a screen asking you to press a button to enter the MOK manager EFI utility. Note that any external external keyboards won't work in this step. Select Enroll MOK in the first menu, then continue, and then select Yes to enroll the keys, and re-enter the password established in the previous step. Then select OK to continue the system boot.

Steps:
  -> "Enroll MOK"
  -> "Continue".
  -> "Yes".
  -> Enter the password you set up just now.
  -> Select "OK" and the computer will reboot again.

5. Verify if your key "MODULE" is loaded and enrolled.


  $ sudo mokutil --test-key /var/lib/shim-signed/mok/MOK.der
  $ sudo dmesg | grep cert
  $ sudo cat /proc/keys | grep MODULE
  $ openssl x509 -in /var/lib/shim-signed/mok/MOK.der -inform DER -text
  -noout

6. Sign the module with sign-file

Use the same password you used before when setting up MOK for the BIOS to avoid confusion. Make sure you type the password carefully here with no errors, and don't get confused by it just waiting.


  $ sudo su
  ~# read -s KBUILD_SIGN_PIN

Next export it and sign all modules.


  $ sudo su
  ~# export KBUILD_SIGN_PIN

NOTE: KBUILD_SIGN_PIN allows a passphrase or PIN to be passed to the sign-file utility when signing kernel modules, if the private key requires such.

For signing the module, depending on your platform, the exact location of sign-file might vary. In Debian 12 (Bookworm) it was in kernel generic /usr/src/linux-kbuild-$(uname -r | cut -d . -f 1-2)/scripts/sign-file.

And where was the module installed? In /lib/modules/$(uname -r)/kernel/drivers/.ko.


  $ sudo modinfo -n rtw_8723d
  /lib/modules/6.1.0-13-amd64/kernel/drivers/net/wireless/realtek/rtw88/rtw_8723d.ko

To sign modules (with your KBUILD_SIGN_PIN), go to the directory containing the modules, and run


  $ sudo su
  ~# cd
  /lib/modules/6.1.0-13-amd64/kernel/drivers/net/wireless/realtek/rtw88/
  ~# /usr/src/linux-kbuild-6.1/scripts/sign-file sha256
  /var/lib/shim-signed/mok/MOK.priv /var/lib/shim-signed/mok/MOK.der
  rtw_8723d.ko

Other not tested form:


  sudo --preserve-env=KBUILD_SIGN_PIN sh /usr/src/linux-kbuild-$(uname
  -r | cut -d . -f 1-2)/scripts/sign-file sha256
  /var/lib/shim-signed/mok/MOK.priv /var/lib/shim-signed/mok/MOK.der
  $(modinfo -n rtw_8723d)

Assuming you type the password correct, you won't get any errors. You should be able to now see that a module is signed. You can pick any module in that directory but as an example:


  $ sudo modinfo rtw_8723d
  (...)
  signer: MODULE
  sig_key:XX:XX:XX:XX:XX:XX:XX:XX...
  sig_hashalgo: sha256
  signature:XX:XX:XX:XX:XX:XX:XX:XX...
  (...)

NOTE: Filename may be different, just use tab completion to find the appropriate file to check some other name.

You could try loading the modules:


  $ sudo modprobe -v rtw_8723d

After any kernel module loading failure, you should check the dmesg output: it might include a more specific error message. In this case, it is likely to indicate that a module signature failed a validity check.


  $ sudo dmesg --since -1m

If the modules are needed to boot your machine, make sure to update the initramfs, e.g. using


  $ sudo update-initramfs -k all -u

Building and signing modules is independent of building and signing your own kernel. To sign a custom kernel or any other EFI binary you want to have loaded by shim (PEM), you’ll need to use a different command: sbsign (PEM). In this case, we’ll need the certificate in a different format, mokutil needs DER, sbsign needs PEM. Convert the certificate into PEM (.der to .pem), for example:


  $ sudo openssl x509 -in MOK.der -inform DER -outform PEM -out MOK.pem

For example, use it to sign our Kernel:


  $ sudo sbsign --key MOK.priv --cert MOK.pem
  "/boot/vmlinuz-$VERSION" --output
  "/boot/vmlinuz-$VERSION.tmp"
  $ sudo mv "/boot/vmlinuz-$VERSION.tmp"
  "/boot/vmlinuz-$VERSION"


  $ sudo openssl x509 -in MOK.der -inform DER -outform PEM -out MOK.pem

For example, use it to sign our Kernel:


  $ sudo sbsign --key /var/lib/shim-signed/mok/MOK.priv --cert
  /var/lib/shim-signed/mok/MOK.pem --output /boot/vmlinuz-$(uname
  -r)-signed /boot/vmlinuz-$(uname -r)
  $ sudo mv /boot/vmlinuz-$(uname -r)-signed /boot/vmlinuz-$(uname -r)

After this, the kernel will be signed and should be able to boot under Secure Boot.

Signing other EFI binaries can follow the same process:


  $ sudo sbsign --key /var/lib/shim-signed/mok/MOK.priv --cert
  /var/lib/shim-signed/mok/MOK.pem --output /path/to/signed-binary
  /path/to/binary

Ensure the signed binaries are in the correct locations and properly referenced in your bootloader configuration.

7. Verify the Signature

To verify the signature on a kernel module or binary, use the following command:


  $ sbverify --cert /var/lib/shim-signed/mok/MOK.pem
  /boot/vmlinuz-$(uname -r)

If the signature is valid, the output will confirm it. Otherwise, you'll get an error message indicating the issue.

8. Troubleshooting Common Issues

Here are some common issues and their solutions:

Invalid Signature: Ensure the correct key and certificate are used, and the signature command is typed correctly without errors.
Modules not loading: Check dmesg logs for specific error messages and verify the Secure Boot state and MOK enrollment.
Secure Boot disabled: Re-enable Secure Boot in your BIOS/UEFI settings.

3.09 Sign NVIDIA

https://wiki.debian.org/DontBreakDebian#Don.27t_use_GPU_manufacturer_install_scripts
https://github.com/NVIDIA/open-gpu-kernel-modules
https://askubuntu.com/questions/1023036/how-to-install-nvidia-driver-with-secure-boot-enabled

Download the latest driver from the NVIDIA website: https://geforce.com/drivers.

Create a new pair of private key (Nvidia.key) and public key (Nvidia.der) by running the command:


  openssl req -new -x509 -newkey rsa:2048 -keyout PATH_TO_PRIVATE_KEY
  -outform DER -out PATH_TO_PUBLIC_KEY -nodes -days 36500 -subj
  "/CN=Graphics Drivers"

Example:


  openssl req -new -x509 -newkey rsa:2048 -keyout /home/$USER/Nvidia.key
  -outform DER -out /home/$USER/Nvidia.der -nodes -days 36500 -subj
  "/CN=Graphics Drivers"

Enroll the public key (nvidia.der) to MOK (Machine Owner Key) by entering the command:

sudo mokutil --import
  PATH_TO_PUBLIC_KEY

Example:

sudo mokutil --import
  /home/$USER/Nvidia.der

This command requires you to create a password for enrolling. Afterwards, reboot your computer, in the next boot, when the system asks you to enroll, you enter the password you created in this step to enroll it. Read more: https://sourceware.org/systemtap/wiki/SecureBoot

For installing the NVidia driver for the first time, you need to disable the Nouveau kernel driver by entering the command:

echo options nouveau modeset=0 | sudo tee -a
  /etc/modprobe.d/nouveau-kms.conf; sudo update-initramfs -u

Reboot.

Install the driver by entering the command:

sudo sh
  ./XXXXXX.run -s --module-signing-secret-key=PATH_TO_PRIVATE_KEY
  --module-signing-public-key=PATH_TO_PUBLIC_KEY

here:

XXXXXX: name of file installer (downloaded from NVIDIA).

PATH_TO_PRIVATE_KEY: full path to private key. If you place it in your home folder, use /home/USER_NAME/ instead of ~.

PATH_TO_PUBLIC_KEY: full path to public key. If you place it in your home folder, use /home/USER_NAME/ instead of ~.

Example:

sudo sh ./NVIDIA-Linux-x86_64-390.67.run -s
  --module-signing-secret-key=/home/$USER/Nvidia.key
  --module-signing-public-key=/home/$USER/Nvidia.der

Done.

3.10 Sign VirtualBox

How to get VirtualBox signed for Secure Boot

https://wiki.debian.org/SecureBoot#MOK_-_Machine_Owner_Key

3.11 Sign Ventoy

About Secure Boot in UEFI mode

https://ventoy.net/en/doc_secure.html

3.12 rEFInd Bootloader

https://rodsbooks.com/refind/getting.html
https://wiki.ubuntu.com/EFIBootLoaders

3.13 Sign Custom Secure Keys

https://github.com/nsacyber/Hardware-and-Firmware-Security-Guidance/blob/master/secureboot/Linux.md

3.14 Sign with TPM 2.0

https://github.com/squarooticus/efi-measured-boot
https://github.com/osresearch/safeboot

3.15 Secure Boot with Yubikey

https://github.com/DimanNe/secure-boot
https://github.com/sandrokeil/yubikey-full-disk-encryption-secure-boot-uefi

4. SYSTEM SECURITY

👷🛠️UNDER CONSTRUCTION🚧🏗

4.01 Introduction

General Strategies for Protection from Totalitarian and Police States

Maintain a Low Profile:
- Avoid drawing attention to yourself or your activities.
- Do not openly criticize the government or its officials.
- Dress and behave in ways that do not attract undue attention.
Stay Informed:
- Listen to news from foreign radio stations or other underground sources to stay informed about political developments and potential threats.
- Share information discreetly with trusted friends and family.
Create and Use Safe Spaces:
- Establish safe meeting places where you can discuss sensitive matters.
- Use homes of trusted friends or remote locations to avoid surveillance.
Build Trusted Networks:
- Develop and maintain a close-knit group of trusted friends and family.
- Form or join underground resistance groups if necessary.
Secure Communication:
- Use coded language or signals to communicate sensitive information.
- Write letters using invisible ink or steganography.
- Use face-to-face communication in secluded areas to avoid interception.
Document Security:
- Keep important documents hidden or disguised.
- Destroy incriminating evidence that could be used against you or others.
- Use false identities or forged documents when necessary.
Travel Discreetly:
- Avoid traveling frequently or taking predictable routes.
- Use aliases and travel documents that do not reveal your true identity.
Financial Caution:
- Avoid transactions that could be traced or arouse suspicion.
- Use cash and barter systems rather than banks or official financial channels.
Prepare for Searches:
- Keep your home clean of any incriminating materials.
- Have hiding places for important documents or objects.
- Be prepared to answer questions convincingly if interrogated.
Learn Self-Defense and Survival Skills:
- Acquire basic self-defense skills to protect yourself physically.
- Learn survival skills in case you need to flee or live off the grid.
Plan Escape Routes:
- Have a plan for quickly leaving your home or country if necessary.
- Keep emergency supplies and documents ready for a quick departure.
Utilize Community Support:
- Engage in mutual aid with trusted community members.
- Support and be supported by others facing similar threats.
Stay Physically and Mentally Fit:
- Maintain your health to endure physical and mental stress.
- Practice stress-relief techniques and maintain a positive mindset.
Observe and Adapt:
- Stay alert to changes in the political environment and adapt accordingly.
- Learn from the experiences of others who have successfully evaded state scrutiny.
Document and Share Truths Discreetly:
- Keep a record of injustices and human rights abuses.
- Share these records with trusted sources, both locally and internationally, when safe to do so.

Additional Strategies

Stay Silent: avoid discussing sensitive information, especially online or in public.
Keep Personal and Professional Separate: Don't mix your personal life with any potentially sensitive or illegal activities.
Avoid Bragging: don’t boast about your activities, connections, or capabilities, as this can draw unwanted attention.
Avoid contaminating ("watering hole"): refrain from contamination of systems, maintain strict compartmentalization.
Keep Operational Details Confidential: never disclose specific details about your operations or plans to anyone.
Maintain OpSec: operational Security is crucial; don't reveal your plans, strategies, or tactics.
Be Mindful of Metadata: understand that even seemingly harmless metadata (e.g., location data, timestamps) can be used to track your activities, so be cautious about what information you share and how it can be interpreted.
Exercise Caution with Trust: be wary of trusting anyone, and never allow yourself to be blackmailed.
Operate Discreetly: avoid conducting any sensitive activities from your home or easily traceable locations.
Cultivate Safe Habits: practice cautious behaviors online and offline to minimize risks.
Understand Human Vulnerabilities: remember that humans can be the weakest link in security, so be mindful of social engineering tactics.
Know Your Rights: familiarize yourself with your legal rights and protections regarding surveillance and privacy (search warrant rules), so you can assert them if necessary.
Stay Informed about Surveillance Practices: educate yourself about common surveillance techniques used by law enforcement and intelligence agencies, so you can take appropriate precautions to protect your privacy.
Seek Specialized Advice if Needed: if you believe you are being unfairly targeted or monitored, seek legal advice from a qualified attorney who specializes in privacy and surveillance issues. Report suspicious behavior or incidents to journalists or tech NGOs.

*Social engineering tactics and contaminated systems (watering hole): It seems that the worst problem is collateral intrusion, the involvement of seemingly trustworthy uninvolved close third parties. The end of trust.

General guidelines for protecting systens

Install and Update Security Software

Use reputable antivirus software compatible with Linux, such as ClamAV.
Update your antivirus definitions regularly to detect new threats.

Secure System Updates

Set up automatic updates for your Linux distribution to ensure you receive security patches promptly.
Regularly update installed software and libraries to the latest versions.

Enable Firewall

Configure a firewall, such as UFW (Uncomplicated Firewall) or iptables, to control incoming and outgoing network traffic.
Define strict rules to block unauthorized access and suspicious connections.

Secure Network Connections

Use encrypted connections (HTTPS, SSH, VPN) for sensitive activities and avoid public or unsecured networks.
Disable unnecessary network services and ports to reduce the attack surface.

Implement User Permissions

Use non-administrator (regular) user accounts for daily tasks to limit the impact of malware if a compromise occurs.
Avoid running applications with root privileges unless absolutely necessary.

Secure File Systems

Enable filesystem encryption (e.g., using LUKS for disk encryption) to protect sensitive data.
Set appropriate file permissions to restrict access to critical system files and directories.

Monitor System Logs

Regularly review system logs (e.g., /var/log/syslog, /var/log/auth.log) for unusual activities or potential security incidents.
Install and configure intrusion detection systems (IDS) or security monitoring tools.

Backup and Recovery

Implement regular backups of important data to an external device or secure cloud storage.
Test your backup and recovery procedures periodically to ensure they work effectively.

Hardening Services

Disable or remove unnecessary services and daemons to reduce the attack surface.
Configure services securely, following best practices and guidelines for each service.

Stay Informed and Educated

Stay updated with cybersecurity news, vulnerabilities, and threat intelligence related to Linux systems.
Educate yourself on Linux security best practices and techniques for detecting and mitigating malware.

Utilize Security Tools

Use tools like Lynis, OpenVAS, or Nessus for vulnerability scanning and system hardening.
Consider employing intrusion prevention systems (IPS) or endpoint security solutions for additional protection.

Secure Boot and BIOS/UEFI

Enable Secure Boot in your system's BIOS/UEFI settings to ensure that only trusted software can boot.
Regularly update your BIOS/UEFI firmware to patch known vulnerabilities.

4.02 Apparmor

https://github.com/Kicksecure/security-misc
https://apparmor.net
https://wiki.debian.org/AppArmor
https://wiki.debian.org/AppArmor/HowToUse
https://github.com/Kicksecure/apparmor-profile-torbrowser
https://wiki.ubuntu.com/DebuggingApparmor

$ sudo apt install -y apparmor
  &&

Copy

$ sudo apt install -y
  apparmor-profiles

Copy

$ sudo apt install -y
  apparmor-utils

Copy

$ sudo apt install -y
  apparmor-profiles-extra

Copy

*Note: an AppArmor rule could prevent port use by an individual program.

4.03 Autentication

How to securely configure sudo for everyday use on your personal Linux machine

Open a Terminal Window:
- Press Ctrl + Alt + T or search for "Terminal" in your application menu.
Switch to the Root User:
- If you have access to the root account, switch to root by running:
```
  $ su -
```
  Copy
- Enter the root password when prompted.
Run visudo:
- As the root user, execute:
```
# visudo
```
  Copy
- This command opens the sudoers file in your default text editor and ensures that any syntax errors won't be saved, preventing potential misconfigurations.
Add Your User to the Sudoers File:
- Locate the line that includes root ALL=(ALL:ALL) ALL .
- Below that line, add the following line, replacing your_username with your actual username:
```
  $ your_username ALL=(ALL:ALL) ALL
```
  Copy
- Save the file and exit the editor:
  - For ano , use Ctrl + O to save and press Enter, press Ctrl + X to exit.

Test Sudo Access

Open a New Terminal Window or Log Out and Log Back In:
- This ensures that the changes take effect.
Test Sudo Access:
- Run a command with sudo to verify:
```
  $ sudo ls
```
  Copy
- Enter your password when prompted.
- If the command executes successfully, sudo is configured correctly for your user.

Optional: Configure Sudo Timeout

Open the Sudoers Configuration for Timeout:
- Create or edit a specific sudoers configuration file for the timeout setting:
```
  $ sudo nano /etc/sudoers.d/timeout
```
  Copy
Set the Timeout Value:
- Add or modify the following line to set the timeout (e.g., 3 minutes):
```
Defaults timestamp_timeout=3
```
  Copy
- Save the file and exit nano.

Additional Tips

Use Sudo Only When Necessary:
- Only use sudo for commands that require elevated privileges to minimize the risk of accidental system changes.
Keep Your System Updated Regularly:
- Run updates frequently to ensure you have the latest security patches and software versions.
Secure Your Password:
- Use a strong and unique password for your user account to enhance security.

Note: Logoff to take effect.

Note: It's worthless if full disk encryption with strong password is not implemented.

Note: To avoid having your password stored in the bash history or typed directly into the command line:

$ read -s -p "Enter your sudo password:"
  password; echo "$password" | sudo -S su -c "/etc/init.d/iptables -L
  -vn"; unset password

Linux Permissions and Ownership Table

Permissions Table

Permission string	Octal code	Meaning
rwxrwxrwx	777	Read, write, and execute permissions for all users.
rwxr-xr-x	755	Read and execute permission for all users. The file's owner also has write permission.
rwxr-x---	750	Read and execute permission for the owner and group. The file's owner also has write permission. Users who aren't the file's owner or members of the group have no access to the file.
rwx------	700	Read, write, and execute permissions for the file's owner only; all others have no access.
rw-rw-rw-	666	Read and write permissions for all users. No execute permissions for anybody.
rw-rw-r--	664	Read and write permissions for the owner and group. Read-only permission for all others.
rw-rw----	660	Read and write permissions for the owner and group. No world permissions.
rw-r--r--	644	Read and write permissions for the owner. Read-only permission for all others.
rw-r-----	640	Read and write permissions for the owner, and read-only permission for the group. No permission for others.
rw-------	600	Read and write permissions for the owner. No permission for anybody else.
r--------	400	Read permission for the owner. No permission for anybody else.

Ownership Table

Owner	Group	Description
User	Group	The owner of the file, typically the user who created the file. They have the permissions granted by the user part of the permission string.
Group	Group	Users who are part of the group associated with the file. They have the permissions granted by the group part of the permission string.
Others	Others	All other users who are not the owner or part of the group associated with the file. They have the permissions granted by the others part of the permission string.

Time-based Autentication (OTP, TOTP, and HOTP)

Term	Definition	Example
OTP	One-Time Password	A unique password that is valid for only one login session or transaction.
TOTP	Time-Based One-Time Password	An OTP that is valid for a certain period of time, typically 30 seconds.
HOTP	HMAC-Based One-Time Password	An OTP that is generated based on a counter and a shared secret key.

Nuke Password

$ sudo apt install
  cryptsetup-nuke-password

Copy

4.04 Audit System

https://redhat.com/sysadmin/configure-linux-auditing-auditd

System Auditors

Automatic

$ sudo apt install lynis

Copy

$ sudo apt install checksecurity

Copy

Manual

$ sudo apt install audit

Copy

Rootkit Detect

$ sudo apt install chkrootkit

Copy

$ sudo apt install rkhunter

Copy

$ sudo apt install chkboot

Copy

4.05 Antimalware

Dangerous PDFs, Office Documents, or Images

Handle potentially dangerous PDFs, office documents, or images by converting them to safe PDFs.

Dangerzone
GitHub - Freedom of Press

Online Virus and URL Scanning

Online Phishing and Email Reputation Verifiers

ClamAV

ClamAV is an open-source (GPL) antivirus toolkit designed for UNIX/Linux systems. It is primarily used on file/mail servers for scanning and detecting malware.

Key points about ClamAV:

It detects malware but does not typically disinfect files; it isolates or removes them.
It primarily targets Windows viruses and malware using its built-in signatures.
False positives can occur, so using the --remove option during scans is generally not recommended.

Installing ClamAV

ClamAV (GUI)

To install the ClamAV GUI:

$ sudo apt install clamtk

ClamAV (CLI)

To install ClamAV and related components:

$ sudo apt install -y clamav

Copy

$ sudo apt install -y clamav-daemon

Copy

$ sudo systemctl start clamav-freshclam

Copy

$ sudo freshclam

Copy

*Note: Clamscan doesn't require the daemon to be running.

ClamAV Configuration and Commands

Configuring ClamAV Daemon

To optimize the ClamAV daemon configuration:


  $ sudo nano /etc/clamav/clamd.conf
        ConcurrentDatabaseReload no
        ReadTimeout 10
        MaxThreads 3


  $ sudo nano /etc/systemd/system/clamav-daemon.service.d/extend.conf
        [Service]
        IOSchedulingPriority=7
        CPUSchedulingPolicy=5
        MemoryLimit=256M
        CPUQuota=30%
        Nice=19

Using ClamAV Commands

  Basic Commands:
  $ man clamscan
  $ mkdir /home/$USER/Virus && touch /home/$USER/Virus/Virus.txt
  $ sudo freshclam && sudo clamscan -vir --copy=/home/$USER/Virus --log=/home/$USER/Virus/Virus.txt --exclude-dir="^/sys" /usb

  Scan a File:
  $ clamscan --verbose /file.bin

  Scan Compressed Files:
  $ clamscan --verbose --scan-archive --alert-encrypted /file.zip

  Other Commands:
  $ clamscan -vr --suppress-ok-results --bell /$USER/home
  $ clamscan -vro --heuristic-alert --copy=/home/$USER/Virus --log=/home/$USER/Virus/Virus.txt --bell /$USER/home
  $ clamscan -vro --bell --remove /$USER/home

Debugging and Maintenance

  Debugging:
  $ sudo cat /etc/clamav/clamd.conf
  $ sudo nano /etc/clamav/clamd.conf

  $ sudo nano /etc/systemd/system/clamav-daemon.service.d/extend.conf

  $ sudo cat /var/log/clamav/clamav.log

  $ sudo systemctl status clamav-daemon
  $ sudo systemctl stop clamav-daemon
  $ sudo systemctl disable clamav-daemon

  $ sudo systemctl status clamav-freshclam
  $ sudo systemctl stop clamav-freshclam
  $ sudo systemctl disable clamav-freshclam

  $ sudo crontab -l
  $ sudo systemctl list-timers

*If you encounter AppArmor denials related to clamd, switch the profile to complain-only mode:

$ sudo aa-complain clamd

Signatures Compatible with ClamAV

RFXN Linux Malware Detect by RFXN
Malware Blocklist Malware Blocklist
Malware Analysis with ClamAV and YARA
YARA Rules YARA Rules for ClamAV
https://github.com/reversinglabs/reversinglabs-yara-rules
https://github.com/bartblaze/Yara-rules
https://github.com/0xThiebaut/Signatures

$ sudo apt install yara

Yara rules samples:


  $ sudo nano /etc/yara/rules.yar
  $ yara /etc/yara/rules.yar /path/to/scan


    // Example YARA rule 1
  rule detect_malware {
    strings:
      $a = "malicious_string"
    condition:
      $a
  }


  // Example YARA rule 2
  rule MemoryModule {
      meta:
  	id = "6O9mUMvPhziJ72IXHf6muZ"
  	fingerprint = "4aa0a23f28698898404d700cb363ddf06dd275f5798815e797113656a2a40ae8"
  	version = "1.0"
  	date = "2020-05-06"
  	modified = "2020-05-06"
  	status = "RELEASED"
  	sharing = "TLP:WHITE"
  	source = "CCCS"
  	author = "analyst@CCCS"
  	description = "Yara rule to detect usage of MemoryModule Library"
  	category = "TECHNIQUE"
  	technique = "LOADER:MEMORYMODULE"
  	mitre_att = "T1129"
  	report = "TA20-0192"
  	hash = "812bbe8b9acabad05b08add50ee55c883e1f7998f3a7cae273d3f0d572a79adc"

      strings:
          $func_ptr =    {55 8B EC 6A 00 68 [3] 00 68 [3] 00 68 [3] 00 68 [3] 00 68 [3] 00}
          $func_ptr_64 = {48 [3] 48 [4] 00 00 00 00 48 8? [5] 48 8? [3] 4? 8? [5] 48 8? [3-5] 48 8?}
          $api_1 = "LoadLibraryA"
          $api_2 = "GetProcAddress"
          $api_3 = "FreeLibrary"
          $api_4 = "VirtualFree"
          $api_5 = "VirtualProtect"
          $api_6 = "VirtualAlloc"

      condition:
          uint16(0) == 0x5a4d and all of ($api*) and ($func_ptr or $func_ptr_64)
  }

ClamAV with Heuristic and Behavioral Analysis Tools

ClamAV is a powerful open-source antivirus toolkit primarily focused on signature-based detection. To enhance its capabilities with heuristic and behavioral analysis, we can integrate it with additional tools that provide these features.


  $ sudo apt update
  $ sudo apt install -y clamav clamav-daemon

1. Start the ClamAV daemon and update the virus definitions:


  $ sudo systemctl start clamav-daemon
  $ sudo freshclam

2. Installing Tools for Heuristic and Behavioral Analysis

YARA Rules

YARA is a powerful tool for pattern matching and analyzing malware characteristics. Install YARA and its development tools:

$ sudo apt install -y yara yara-python

Create and use YARA rules for enhanced detection:


  $ sudo nano /etc/yara/rules.yar
  // Example YARA rule
  rule detect_malware {
    strings:
      $a = "malicious_string"
    condition:
      $a
  }

  $ yara /etc/yara/rules.yar /path/to/scan

Cuckoo Sandbox

Cuckoo Sandbox provides dynamic analysis capabilities. Install and configure Cuckoo Sandbox:


  $ sudo apt install -y python3 python3-pip virtualenv
  $ git clone https://github.com/cuckoosandbox/cuckoo
  $ cd cuckoo
  $ virtualenv venv
  $ . venv/bin/activate
  $ pip install -r requirements.txt
  $ cuckoo init

Start Cuckoo's services and analyze files:

$ cuckoo -d

Integrating ClamAV with Heuristic and Behavioral Analysis

Configure ClamAV to use YARA rules for additional scanning:


  $ sudo nano /etc/clamav/clamd.conf
  # Add or modify the following line
  Include /etc/yara/rules.yar

  $ sudo systemctl restart clamav-daemon

For dynamic analysis with Cuckoo Sandbox, configure ClamAV as a scanner:


  $ sudo nano /etc/cuckoo/conf/processing.conf
  [scan]
  enabled = yes
  antivirus = clamav

Ensure Cuckoo is configured to forward samples to ClamAV for scanning:


  $ sudo nano /etc/cuckoo/conf/reporting.conf
  [clamav]
  enabled = yes

3. Running Scans and Analyzing Results

Run ClamAV scans combined with heuristic and behavioral analysis tools:


  $ sudo clamscan -r --detect-pua /path/to/scan
  $ yara /etc/yara/rules.yar /path/to/scan
  $ cuckoo submit /path/to/sample

ESET NOD32 Antivirus for Linux Desktop

For users preferring a desktop antivirus solution:

Download ESET NOD32 Antivirus for Linux Desktop

4.06 Updating

Setting Up Automatic Updates on Debian

Debian Handbook - Regular Upgrades

Apply security updates as quickly as possible. According to a 2020 study by Unit 42 at Palo Alto Networks, approximately 80% of exploits are published faster than Common Vulnerabilities and Exposures (CVEs).

Comparison: cron vs systemd-cron

Debian offers two main methods for scheduling automatic updates: cron and systemd-cron. Each has its own advantages:

cron:
- Traditional Unix tool.
- Uses crontab files for scheduling.
- Suitable for simple recurring tasks.
- Being a simpler tool, it lacks some of the advanced security features inherent in systemd, such as sandboxing of services and strict resource management.
systemd-cron:
- Part of systemd with tighter integration.
- Uses timer units (.timer files).
- Allows for more complex scheduling and dependency management.
- Benefits from systemd's security features like service sandboxing (SystemCallFilter, ProtectSystem, etc.), user/service resource control (CPUShares, MemoryLimit, etc.), and control group (cgroup) management.

Set up automatic updates on Debian

Here are alternatives to set up automatic updates using both cron or systemd-cron:

Using cron:

Update Hourly:

  0 * * * * /usr/bin/apt-get update && /usr/bin/apt-get -y upgrade

Update on Sundays:

  0 4 * * 0 /usr/bin/apt-get update && /usr/bin/apt-get -y upgrade

Using systemd-cron:

Update Hourly:

Create a systemd timer unit (update-hourly.timer):

  [Unit]
  Description=Run apt update and upgrade every hour
  
  [Timer]
  OnCalendar=*:00:00
  Persistent=true
  
  [Install]
  WantedBy=timers.target

Create a corresponding systemd service unit (update-hourly.service):

  [Unit]
  Description=Run apt update and upgrade every hour
  
  [Service]
  Type=oneshot
  ExecStart=/usr/bin/apt-get update && /usr/bin/apt-get -y upgrade

Update on Sundays:

Create a systemd timer unit (update-sundays.timer):

  [Unit]
  Description=Run apt update and upgrade every Sunday
  
  [Timer]
  OnCalendar=Sun *-*-* 04:00:00
  Persistent=true
  
  [Install]
  WantedBy=timers.target

Create a corresponding systemd service unit (update-sundays.service):

  [Unit]
  Description=Run apt update and upgrade every Sunday
  
  [Service]
  Type=oneshot
  ExecStart=/usr/bin/apt-get update && /usr/bin/apt-get -y upgrade

4.07 Encryption

Encryption

👷🛠️UNDER CONSTRUCTION🚧🏗

Visit our repo tree: 02_SECURITY/03_Cryptography_and_Passwords

Visit our repo tree: 01_INITIAL_SETUP/04_Custom_System/03_Debootstrap

VeraCrypt Anti-Tampering Measures

• Disk Encryption

∙ ZuluCrypt (GUI)

https://mhogomchungu.github.io/zuluCrypt
https://github.com/mhogomchungu/zuluCrypt

$ sudo apt install zulucrypt-gui

Copy

∙ VeraCrypt (GUI)

https://veracrypt.fr/en/Downloads.html https://github.com/veracrypt/VeraCrypt https://reddit.com/r/VeraCrypt

Command to automount favorite volume at startup session:

/usr/bin/veracrypt %f /dev/sda2

∙ SiriKali (GUI)

https://mhogomchungu.github.io/sirikali

Tamper-Proof Encryption with VeraCrypt

Step 1: Create a VeraCrypt Volume

Open a terminal and create a new VeraCrypt volume:

$ veracrypt --create /path/to/your_volume.tc

Copy

Follow the prompts to set up the volume:

Select the volume type (Standard or Hidden).
Specify the volume size.
Choose the encryption algorithm and hash algorithm.
Set the password for the volume.
Format the volume with the desired filesystem.

Step 2: Mount the VeraCrypt Volume

To mount the volume, use the following command:

$ sudo veracrypt /path/to/your_volume.tc /mnt/veracrypt1

Copy

You will be prompted to enter the password for the volume.

Step 3: Copy Data to the VeraCrypt Volume

After mounting the volume, you can copy the data you want to protect into the mounted directory:

$ sudo cp -r /path/to/your_data /mnt/veracrypt1/

Copy

Step 4: Unmount the VeraCrypt Volume

Once you have copied your data, unmount the volume to ensure it is securely encrypted:

$ sudo veracrypt -d /mnt/veracrypt1

Copy

Step 5: Set Up Tamper-Proof Mechanisms

To make the encryption tamper-proof, you can utilize hash functions to detect any unauthorized modifications. Follow these steps:

Create a SHA256 hash of the encrypted volume:

$ sha256sum /path/to/your_volume.tc > /path/to/your_volume.tc.sha256

Copy

Store the hash securely. This hash will be used to verify the integrity of the encrypted volume.

Step 6: Verify the Integrity of the Encrypted Volume

Whenever you want to ensure that the volume has not been tampered with, you can verify its hash:

$ sha256sum -c /path/to/your_volume.tc.sha256

Copy

If the output says OK, the volume is intact. Otherwise, it has been tampered with.

Step 7: Automount the Volume at Startup (Optional)

If you want to automount the volume at startup, create a script with the following content:


  #!/bin/bash
  veracrypt --text --non-interactive --password='your_password' --mount /path/to/your_volume.tc /mnt/veracrypt1

Copy

Save the script, make it executable, and set it to run at startup:

$ chmod +x /path/to/your_script.sh

Copy

$ sudo crontab -e

Copy

Add the following line to the crontab file to run the script at startup:

@reboot /path/to/your_script.sh

Copy

Step 8: Secure the Script

Ensure that only root or authorized users can read the script by setting appropriate permissions:

$ sudo chown root:root /path/to/your_script.sh

Copy

$ sudo chmod 700
  /path/to/your_script.sh

Copy

Passwordless VeraCrypt Setup

To set up VeraCrypt to run without requiring a password for each operation, follow these steps:

Step 1: Create a VeraCrypt Group

Create a new group called veracrypt:

$ sudo groupadd veracrypt

Copy

Step 2: Add Your User to the VeraCrypt Group

Add your user to the veracrypt group. You can use either of the following commands:

$ sudo usermod -aG veracrypt $USER

Copy

Step 3: Edit the Sudoers File

Edit the sudoers file to grant passwordless execution rights for VeraCrypt to the veracrypt group:

$ sudoedit /etc/sudoers

Copy

Add the following lines to the sudoers file:


  # Allow members of group veracrypt to run VeraCrypt commands as root without a password
  %veracrypt ALL=(root) NOPASSWD:/usr/bin/veracrypt
  # Allow members of group sudo to execute any command
  %sudo ALL=(ALL:ALL) ALL

Save and close the sudoers file after making these changes.

Step 4: Reboot

Finally, reboot your system to apply the changes:

$ sudo reboot

Copy

Archive Encryption

∙ GnuPG - GNU Privacy Guard

  Commands for gnupg
  • How to encrypt file
  $ gpg -c backup.tar.gz
  • How to decrypt file
  $ gpg backup.tar.gz.gpg

7Z

  $ 7z a -p -mhe=on -scrc=AES256 output input

• Cloud Encryption

∙ Cryptomator (GUI)

https://cryptomator.org https://github.com/cryptomator/cryptomator https://github.com/cryptomator/cli https://reddit.com/r/Cryptomator

∙ Duplicati (GUI)

https://duplicati.com https://github.com/duplicati/duplicati https://forum.duplicati.com https://reddit.com/r/duplicati

4.08 Sanitation

Visit our repo tree: /02_SECURITY/04_Backup_and_Sanitization/02_Data_Sanitization

Full Disk Sanitation

Note:

Not all SSDs support sanitize. To properly erase an SSD, use the manufacturer's software. Other methods might not work due to wear leveling and over-provisioning.
If you use SSDs, enable TRIM in your BIOS. Confirm you are using an SSD in the BIOS options.
Consider potential hardware flaws and always back up your data before performing any erase operation.
Sanitizing an SSD can help in maintaining its performance and security but improper methods can lead to data being recoverable.
Ensure that the SSD firmware is up-to-date before performing any sanitation procedure to avoid issues.

Manufacturers that Supply Software to Update Firmware and Perform Tasks Like Secure Erase

Additional Tools for Disk Sanitation

ShredOS

ShredOS - A USB bootable GNU/Linux environment for full disk sanitation.

Nwipe

Nwipe - A powerful tool for securely erasing disks.

$ sudo apt install -y nwipe

Copy

Hdparm

hdparm is a command-line utility to set and view hardware parameters of hard disk drives, including SSDs.

It can also be used for secure erasure of SSDs that support the ATA Secure Erase command. However, not all SSDs fully support this command, and using manufacturer-provided tools is generally recommended for secure erasure.

Before attempting to use hdparm for secure erasure, verify the SSD's capabilities and consult the manufacturer's documentation.

$ sudo apt install -y hdparm

Copy

Using Hdparm for Secure Erase

To perform a secure erase on an SSD using hdparm, follow these steps:


  $ sudo hdparm --user-master u --security-set-pass password /dev/sdX

Copy


  $ sudo hdparm --user-master u --security-erase password /dev/sdX

Copy

Replace /dev/sdX with the appropriate device identifier for your SSD and password with a password of your choice. Note that this will erase all data on the drive and it will be unrecoverable.

Verifying TRIM Support

To ensure TRIM is enabled, you can use the following command:

$ sudo fstrim -v /

Copy

This command will trim the filesystem mounted on the root directory. For other filesystems, replace / with the appropriate mount point.

System Sanitation

∙ BleachBit

BleachBit is a powerful open-source tool for cleaning and optimizing your system. It helps in freeing up disk space, maintaining privacy, and improving system performance by securely removing unnecessary files and traces of your activities.

SSD Specifics: SSDs handle data differently from traditional hard drives due to wear leveling and over-provisioning. The TRIM command helps manage unused data blocks, but recovery of deleted files can still be possible if the data has not been overwritten by the SSD's internal processes.

$ sudo apt install bleachbit

Copy

To prevent recovery:

Launch BleachBit as both a user and root.
Go to Options - Preferences - General Tab.
Check "Overwrite contents of files to prevent recovery".

∙ Free Space Erase from CLI

BleachBit can be used from the command line interface (CLI) to perform targeted cleaning operations, including erasing free disk space. First, list available cleaning targets:

$ sudo bleachbit -l

Copy

To securely erase free space across various system areas, use the following command:

$ sudo bleachbit --clean \
      system.cache \
      system.clipboard \
      system.custom \
      system.desktop_entry \
      system.free_disk_space \
      system.localizations \
      system.memory \
      system.recent_documents \
      system.rotated_logs \
      system.tmp \
      system.trash

Copy

Note: This command will permanently remove data from these system areas, ensuring it cannot be recovered.

*Cronjob

Freeze Bug - Free space erase option

Take care with free space erase in root mode, this has several problems. This can block the system from starting because the disk is full of randomized files.

Commands to debug if your are freeze

• Acess tty (teletype)

CTRL + ALT + {2,3,4,5,6}

• Delete tmp files in root

User:Password:

$ sudo su

Copy

# ls

Copy

# rm -R tmp*

Copy

# reboot

Copy

• To find the large files in other folders

$ df -h

Copy

$ df -h ~/.cache

Copy

$ sudo df -h /mnt

Copy

$ find ~/.cache -xdev -type f -size +1G

Copy

$ sudo find /root -xdev -type f -size +1G

Copy

$ rm ~/.cache/tmp*

Copy

$ sudo rm /root/tmp*

Copy

∙ Locale Purge

https://wiki.debian.org/ReduceDebian

Mark your preferred language besides en-US

$ sudo apt install -y localepurge

Copy

$ sudo localepurge

Copy

In Bleachbit as Administrator, go to Options -> Preferences -> Languages Tab and mark your preferred language besides en-US.Start cleaning, this may take some time.

Metadata Cleaners

$ sudo apt install exiftool

$ sudo apt install metacam

$ sudo apt install metadata-cleaner

$ sudo apt install poppler-utils

*Note: PDF Sanitize

Exiftool PDF Tags "All metadata edits are reversible. While this would normally be considered an advantage, it is a potential security problem because old information is never actually deleted from the file. (However, after running ExifTool the old information may be removed permanently using the "qpdf" utility with this command: "qpdf --linearize in.pdf out.pdf".)"

  qpdf --sanitize input.pdf cleaned.pdf
  qpdf --linearize --sanitize input.pdf output.pdf

--linearize: Does not remove any content but reorganizes it for optimized loading, focuses on optimizing the file for faster web viewing. Useful when hosting PDFs on websites to ensure users can start reading the document before the entire file is downloaded.

--sanitize: Removes specific types of content like JavaScript and embedded files, focuses on making the file safe by removing potentially harmful content. Useful when sharing PDFs to ensure they do not contain any malicious content that could harm the recipient's system.

Ghostscript

$ sudo apt install gs

Copy

  gs -dSAFER -dBATCH -dNOPAUSE -dNOCACHE -sDEVICE=pdfwrite -sOutputFile=cleaned.pdf -f input.pdf
  gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dPrinted=false -sOutputFile=output.pdf input.pdf

∙ Exiftool

$ sudo apt install exiftool

Copy

  Commands for exiftool basic commands 
  • Remove all metadata from all files possible inside a folder and all its ubfolders without backup (take care, might affect the colors) 
  $ exiftool -v -all:all= -overwrite_original -r /path 
  • Shows metadata: 
  $ exiftool -a -Title input.pdf 
  $ exiftool -G1 -a input.pdf 
  $ exiftool -G1 -s 'input.pdf' | grep '\[PDF\]' 
  $ exiftool -a -Model -ImageSize photo.jpg 
  • Process all files of specified file type (case insensitive extension) 
  $ exiftool -v -Model -ImageSize -ext jpg /path/to/files/ 
  • Recursively process all jpg files under specified directory and sub-directory 
  $ exiftool -v -r -Model -ImageSize -ext jpg /path/to/files/

*To not create a backup in command-line the option is -overwrite_original.

*To not creat a backup in ExifToolGUI, there's menu "Options ">"Don 't backup files when modifying".

PDF Metadata

$ sudo apt install poppler-utils

Copy

Poppler-utils package contains pdfinfo.

$ pdfinfo input.pdf

Copy

$ pdfinfo -meta input.pdf

Copy

$ pdfinfo -js filename.pdf

Copy

∙ Metadata Cleaner

  Commands
  $ metadata-cleaner /path/to/file.png

5. NETWORK

👷🛠️UNDER CONSTRUCTION🚧🏗

5.01 Router

Router Freedom - Open-source routers - Device Neutrality

https://docs.fsfe.org/en/teams/router-freedom-tech-wiki
https://fsfe.org/contribute/spreadtheword#device-neutrality

"There are a number of open-source options for routers that will take even a small consumer router and turn it into a powerful device with enterprise-level capabilities. My personal favorite is DD-WRT, but other popular options include pfSense, OpenWRT, and Tomato. While you can buy pre-flashed devices in some cases (FlashRouters for DD-WRT and Protectli for pfSense), I always encourage you to do it yourself if you’re comfortable to ensure maximum security (and also to be familiar with the update process). Having said all of this, if you are unsure if an open source router is right for you (the wealth of options can be overwhelming to some), I still encourage you to get a router that wasn’t provided by your ISP. Make sure it offers VLANs and VPN capabilities, as we will be using these heavily to protect your home."

https://thenewoil.org/en/guides/quick-start/wifi-guide

Examples of VPN routers and firmwares

Router	Firmware
EdgeRouter and Ubiquiti GL.iNet Netduma Netgear MikroTik Peplink/Pepwave	OpenWRT AsusWRT Merlin DD-WRT DrayTek Vigor OPNsense 19.1 Padavan pfSense 2.4.4 pfSense 2.4.5 pfSense 2.5 Sabai Tomato

https://openwrt.org
https://pfsense.org

Router Guide

https://avoidthehack.com/router-wireless-guide

Change the default router password
Turn off UPnP
Use the router’s firewall capabilities
Use sufficient Wi-Fi encryption
Set a strong Wi-Fi password
Change the Wi-Fi (SSID) name from the default
Hide the Wi-Fi (SSID)
Consider using open-source router firmware
Keep router firmware updated
Keep other software up to date

*These are solutions outside the military level. Forgetting Wi-Fi and using an RJ-45 to USB should be considered for home use.

*Misconfigured DNS settings on a router may lead to the device sending DNS queries to unintended DNS servers.

5.02 Network

Network Managers

NetworkManager (GUI)

$ sudo apt install network-manager-gnome

NetworkManager (CLI)

$ sudo apt install network-manager

  Commands
  • Connecting WiFi manualy with nmcli
  $ nmcli dev status
  $ nmcli radio wifi on
  • List wifi
  $ nmcli dev wifi list
  • Connect 
  $ sudo mcli dev wifi connect SSID(TAB) 
  $ sudo nmcli --ask dev wifi connect SSID(TAB)
  • Disconnecting
  $ nmcli con down NAME
  • Saved ones
  $ ls /etc/NetworkManager/system-connections/

  Commands
  • Editing
  $ nmcli connection edit
  $ nmcli connection edit type wifi
  $ nmcli c edit type vpn
  $ nmcli c up wificonnectionname
  $ nmcli c show wificonnectionname
  $ nmcli connection show
  $ nmcli connection reload
  • Avtivating MAC randomization
  $ nmcli connection modify NAME 802-11-wireless.mac-address-randomization always

  Config files 
  $ sudo ls /etc/NetworkManager/ 
  $ sudo ls /etc/NetworkManager/system-connections/ 
  $ sudo nano /etc/NetworkManager/NetworkManager.conf/mywifiname 
  $ sudo nano /etc/NetworkManager/NetworkManager.conf

Connman

Connman (GUI)

$ sudo apt install connman-ui

Connman (CLI)

$ sudo apt install connman

  Commands
  $

Setting up networking without a network manager

https://unix.stackexchange.com/questions/253030/how-to-setup-network-without-wicd-or-networkmanager

Set up static networking. Configured only wlan0 because of wireless, you just need to skip the wireless related things in it.

Show your interfaces:

$ ip a show

Note the default Ethernet and wifi interfaces:

Looks our Ethernet port is eth0 and WiFi radio is wlan0

$ ip a show | awk '/^[0-9]: /{print $2}'

The output of this command will look something like this:

lo: eth0: wlan0:

Your gateway IP address is found with:

$ sudo route -n

It provides access to destination 0.0.0.0 (everything). Possible it is 192.168.0.1, which is perfectly nominal.

Let’s do a bit of easy configuration in our /etc/networking/interfaces file. The format of this file is not difficult to put together from the man page, but really, you should search for examples first. Plug in your Ethernet port.

Basically, we’re just adding DHCP entries for our interfaces. Above you’ll see a route to another network that appears when I get a DHCP lease on my Ethernet port. Next, add this:

auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp auto wlan0 iface wlan0 inet dhcp

Next, enable and start the networking service:

sudo update-rc.d networking enable

sudo /etc/init.d/networking start

Let’s make sure this works, by resetting the port with these commands:

sudo ifdown eth0

sudo ip a flush eth0

sudo ifup eth0

This downs the interface, flushes the address assignment to it, and then brings it up. Test it out by pinging your gateway IP: ping 192.168.0.1. If you don’t get a response, your interface is not connected or your made a typo.

Let’s “do some WiFi” next! We want to make an /etc/wpa_supplicant.conf file. Consider mine:

etwork={ ssid="CenturyLink7851" scan_ssid=1 key_mgmt=WPA-PSK psk="4f-------------ac" }

Now we can reset the WiFi interface and put this to work:

sudo ifdown wlan0 sudo ip a flush wlan0 sudo ifup wlan0 sudo wpa_supplicant -Dnl80211 -c /root/wpa_supplicant.conf -iwlan0 -B sudo dhclient wlan0

That should do it. Use a ping to find out, and do it explicitly from wlan0, so it gets it’s address first:

ip a show wlan0 | grep "inet" Presumably dhclient updated your /etc/resolv.conf, so you can also do a: ping -I 192.168.0.45 yahoo.com

You’re now running without NetworkManager!

How to reset Network Manager to default?

https://askubuntu.com/questions/637637/how-to-reset-network-manager-to-default

5.03 DNS

DNS Resolution

• The resolv.conf configuration file

https://wiki.debian.org/NetworkConfiguration
https://wiki.debian.org/resolv.conf
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/manually-configuring-the-etc-resolv-conf-file_configuring-and-managing-networking
https://github.com/jonathanio/update-systemd-resolved
https://freedesktop.org/software/systemd/man/systemd.network.html

• The resolvconf program

https://salsa.debian.org/debian/resolvconf

• The openresolv program

An open-source implementation of resolvconf to properly configure DNS and prevent DNS leaks.

https://roy.marples.name/projects/openresolv

• The systemd-resolved service

https://wiki.archlinux.org/title/Systemd-resolved
https://freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html

• dnsmasq

• Avahi

https://wiki.debian.org/Avahi

Router DNS

https://developers.cloudflare.com/1.1.1.1/ip-addresses
https://opendns.com/setupguide
https://docs.fsfe.org/en/teams/router-freedom-tech-wiki

Misconfigured DNS settings on a router may lead to the device sending DNS queries to unintended DNS servers. Verify the DNS of your WAN are set in your router.

Pi-hole®

The Pi-hole® is a DNS sinkhole that protects your devices from unwanted content without installing any client-side software.

https://pi-hole.net
https://docs.pi-hole.net
https://reddit.com/r/pihole

Others

OpenNIC
Dot-Bit : Decentralized DNS using Bitcoin technology

5.04 Firewall

Visit our repo tree: /03_Network_Security/01_Firewall_Configuration

Note that these commands show ports that are in a listening state, but that doesn’t necessarily mean that the ports are open to the internet, because our firewall may be denying connections.

• GUFW (GUI)

https://gufw.org
https://help.ubuntu.com/community/Gufw

$ sudo apt install gufw

• UFW (CLI)

https://launchpad.net/ufw
https://wiki.debian.org/Uncomplicated%20Firewall%20%28ufw%29
https://wiki.archlinux.org/title/Uncomplicated_Firewall
https://manpages.ubuntu.com/manpages/precise/man8/ufw.8.html
https://help.ubuntu.com/community/UFW
https://paulligocki.com/vpn-only-ufw-setup
https://linuxconfig.org/how-to-install-and-use-ufw-firewall-on-linux
https://openvpn.net
https://pypi.org/project/openpyn

$ sudo apt install ufw

∙ Generic UFW configuration (without VPN)

  Commands, basic to install UFW 
  $ sudo apt install ufw 
  $ sudo ufw enable 
  $ sudo ufw status 
  $ sudo nano /etc/default/ufw
  
  
  IPV6=no 
  
  
  $ sudo nano /etc/sysctl.conf
  
  
  net.ipv6.conf.all.disable_ipv6 = 1 
  net.ipv6.conf.default.disable_ipv6 = 1 
  net.ipv6.conf.lo.disable_ipv6 = 1 
  net.ipv6.conf.tun0.disable_ipv6 = 1 
  
  
  $ sudo ufw default deny incoming 
  $ sudo ufw default allow outgoing 
  $ sudo ufw status numbered 
  $ sudo iptables -L --line-numbers 
  $ sudo ufw delete 123 
  $ sudo ufw reload 
  $ sudo reboot

∙ Advanced

• R-fx Networks Projects - https://rfxn.com
• Vuurmuur Firewall - https://vuurmuur.org
• Port Checker - https://portchecker.co

Note: an AppArmor rule could prevent port use by an individual program.

  Commands, some advanced commands
  • Show which ports are listening for connections
  $ ss -tlnp
  • Check for open ports with nmap 
  $ sudo apt install nmap 
  $ sudo nmap localhost
  • Find the name and IP address of your tunnel
  $ ip -o addr | cut -d'\' -f 1
  • Handling DNS queries
  $ apt install tcpdump 
  $ sudo tcpdump -eni any port 53 
  $ sudo tcpdump -eni any port 53 and host 172.27.10.22 
  $ sudo tcpdump -n -i tun0 udp port 53
  • Show Iptables rules 
  $ sudo iptables -L --line-numbers
  • Open TCP SSH PORT for VPN IP only 
  $ sudo ufw allow from 1.2.3.4 to any port 22 proto tcp comment 'Open TCP SSH PORT for VPN IP only'
  • Open TCP Torrent PORT for VPN IP only 
  $ sudo ufw allow in on tun0 from 10.8.0.0/16 to any port 60000 proto tcp comment 'Open TCP Torrent PORT for VPN IP only'
  • Port Forwarding to router 
  $ sudo iptables -A INPUT -m state --state RELATED,ESTABLISHED -p udp --dport 51413 -j ACCEPT
  • For uploading torrent 
  $ sudo iptables -A OUTPUT -p udp --sport 51413 -j ACCEPT 
  $ sudo ufw allow 51413/udp 
  $ sudo iptables -L --line-numbers
  • Reset UFW 
  $ sudo ufw reset
  • Troubles 
  $ sudo apt purge iptables-persistent

∙ Custom application profile

  Commands 
  $ sudo ls /etc/ufw/applications.d/ 
  $ sudo touch /etc/ufw/applications.d/ufw-custom 
  $ sudo nano /etc/ufw/applications.d/ufw-custom
  
  [CustomApp 1 Full]
  title=The first Custom Application
  description=Custom Application Description
  ports=36892|23976|19827
  
  [CustomApp 1 TCP]
  title=The first Custom Application - TPC only
  description=Custom Application Description
  ports=36892,23976,19827/tcp
  
  [CustomApp 1 UDP]
  title=The first Custom Application - UDP only
  description=Custom Application Description
  ports=36892,23976,19827/udp

  • Check if the syntax is correct 
  $ sudo ufw app info "CustomApp 1 Full"
  
  • Create new rule based on this profile 
  $ sudo ufw allow in on tun0 to any app "CustomApp 1 Full"
  
  • Check 
  $ sudo ufw status numbered | grep CustomApp

∙ Configure NAT with UFW

  Commands 
  $ sudo nano /etc/default/ufw
  DEFAULT_FORWARD_POLICY="ACCEPT" 
  $ sudo nano /etc/ufw/sysctl.conf
  net/ipv4/ip_forward=1 
  $ sudo nano /etc/ufw/before.rules 
  # NAT table rules 
  *nat 
  :POSTROUTING ACCEPT [0:0] 
    # Forward traffic through eth0 - Change to match you out-interface 
  -A POSTROUTING -s 192.168.1.0/24 -o eth0 -j MASQUERADE 
    # don't delete the 'COMMIT' line or these nat table rules won't be processed 
  COMMIT 
  $ sudo ufw disable 
  $ sudo ufw enable

∙ Configure Port Forwarding with UFW

  Commands 
  $ sudo nano /etc/default/before.rules
  :PREROUTING ACCEPT [0:0] 
  -A PREROUTING -i eth0 -d 150.129.148.155 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.120:80 
  -A PREROUTING -i eth0 -d 150.129.148.155 -p tcp --dport 443 -j DNAT --to-destination 192.168.1.120:443 
  -A POSTROUTING -s 192.168.1.0/24-d 192.168.1.0/24 -j MASQUERADE 
 
  $ sudo ufw disable 
  $ sudo ufw enable 
  $ sudo ufw allow proto tcp from any to 150.129.148.155 port 80 
  $ sudo ufw allow proto tcp from any to 150.129.148.155 port 443

5.05 VPN

• Buying VPN Services

∙ Choosing the VPN that's right for you - https://ssd.eff.org/en/module/choosing-vpn-thats-right-you
∙ Choosing the best VPN (for you) - https://reddit.com/r/VPN/comments/4iho8e/that_one_privacy_guys_guide_to_choosing_the_best/?st=iu9u47u7&sh=459a76f2

∙ r/vpnrecommendations - https://reddit.com/r/vpnrecommendations

∙ r/VPN - https://reddit.com/r/VPN

∙ r/VPNTorrents - https://reddit.com/r/VPNTorrents

∙ VPN Alert - https://vpnalert.com

∙ VPN-reviews - https://github.com/techlore/VPN-reviews

∙ Mullvad - https://mullvad.net

∙ Mullvad - https://o54hon2e2vj6c7m3aqqu6uyece65by3vgoxxhlqlsvkmacw6a7m7kiad.onion

∙ Private Internet Access (PIA) - https://privateinternetaccess.com

∙ ProtonVPN - https://protonvpn.com

∙ IVPN - https://ivpn.net

∙ AirVPN - https://airvpn.org

∙ VPN.XXX - https://vpn.xxx

∙ Windscribe - https://windscribe.com

∙ ExpressVPN - https://expressvpn.com/vpnmentor1

∙ NordVPN - https://nordvpn.com

• VPN Guides and Tutorials

∙ That One Privacy Site - https://thatoneprivacysite.net/vpn-section
∙ privacytools.io - https://privacytools.io
∙ VPN over SSH - https://wiki.archlinux.org/index.php/VPN_over_SSH

• Creating your own VPN with VPS

• VPN Protocols

5.06 OpenVPN

OpenVPN

https://openvpn.net/community-resources/how-to/
https://wiki.debian.org/OpenVPN
https://debian-handbook.info/browse/stable/sect.virtual-private-network.html
https://debian.org/doc/manuals/securing-debian-manual/vpn.en.html
https://wiki.archlinux.org/index.php/OpenVPN
https://wiki.archlinux.org/index.php/OpenVPN#DNS
https://ubuntu.com/core/docs/networkmanager/configure-vpn
https://community.openvpn.net
https://github.com/OpenVPN/openvpn/tree/master/sample/sample-config-files
https://linuxconfig.org/how-to-run-openvpn-automatically-on-debian-with-a-static-ip-address
https://linuxconfig.org/how-to-encrypt-your-dns-with-dnscrypt-on-ubuntu-and-debian

OpenVPN Sample Configuration Files 
  $ sudo ls /usr/share/doc/openvpn
  $ /usr/share/doc/openvpn/README.Debian.gz

OpenVPN Client Possibilities

OpenVPN + Network Manager (GUI) + Autostart + Autoconnect + Kill Switch OpenVPN + nmcli (CLI) + Autostart + Autoconnect + Kill Switch

*Autoconnect: random server selection.

Installing OpenVPN with NetworkManager (GUI)

You may use graphical VPN tool network-manager UI by providing the key and certificates

  Commands GUI 
  $ sudo apt install network-manager-openvpn-gnome
  $ m-connection-editor

"Find a network connection, open its settings, then under General, enable Automatically connect to VPN. After saving, a secondaries= line is added in that network's configuration file in the [connection] section. It will contain a list of secondary connection UUIDs to be activated. The configuration file is usually /etc/NetworkManager/system-connections/ ."

Import OVPN to NetworkManager in terminal

  Copy the OpenVPN configuration from your VPN provider into /etc/openvpn   Commands nmcli, to easy import 
   $ sudo nmcli connection import type openvpn file /etc/openvpn/client/cc00-myvpn.com_tcp.ovpn
   $ nmcli connection show
   $ nmcli connection up myopvnname
   $ nmcli connection show
   $ ip route
   $ nmcli connection edit type wifi
   $ nmcli c edit type vpn
   $ nmcli c up wificonnectionname
   $ nmcli c show wificonnectionname
   $ nmcli connection show
   $ nmcli connection reload 
   $ sudo service openvpn restart 
   $ sudo systemctl status NetworkManager.service 
   $ sudo systemctl restart NetworkManager.service

Editing OVPN with NetworkManager in terminal

  Config files 
  $ sudo ls /etc/NetworkManager/ 
  $ sudo ls /etc/NetworkManager/system-connections/ 
  $ sudo nano /etc/NetworkManager/NetworkManager.conf/mywifiname 
 
  $ sudo nano /etc/NetworkManager/NetworkManager.conf

Installing OpenVPN (CLI)

  Commands 
  $ sudo apt install resolvconf 
  $ sudo systemctl enable --now resolvconf.service 
  $ sudo apt install openvpn
  • Copy the OpenVPN configuration from your VPN provider into /etc/openvpn 
  $ sudo wget https://vpnprovider.com/openvpn.zip 
  $ sudo unzip openvpn.zip 
  $ sudo rm openvpn.zip
  $ cd /etc/openvpn
  • Instead of .ovpn extension, OpenVPN on Linux uses .conf for config files. Rename them accordingly, you could simply substitute it in the appropriate file name with copy 
  $ sudo cp cc00-myvpn_tcp.ovpn /etc/openvpn/client/client.conf
  • Alternatively, rename and copy in batch 
  $ sudo rename 's/ovpn/conf/' openvpn/*.ovpn 
  $ sudo cp openvpn/* /etc/openvpn

*resolvconf vs. systemd-resolved

*resolvconf vs. openresolv

Basic OpenVPN Connection (Manual connection for test)

  • Basic connection, OpenVPN will ask for a username and password each time you want to connect, and that's not a good headless setup. 
  $ sudo openvpn cc00-myvpn.com_tcp.ovpn Enter Auth Username: Enter Auth Password: (press TAB for o echo)
  • You can autoconnect with saved username and password, create another file in the OpenVPN folder called, auth.txt . Inside that file, put your VPN username on the first line and your password on the second one. 
  $ sudo touch /etc/openvpn/auth.txt 
  $ sudo nano /etc/openvpn/auth.txt user password 
  $ sudo chmod 600 /etc/openvpn/auth.txt
  • You can autoconnect with saved login 
  $ sudo openvpn --config cc00-myvpn.com_tcp.ovpn --auth-user-pass /etc/openvpn/auth.txt (...) Initialization Sequence Completed

Basic connection with autoconnect and DNS resolver, make OpenVPN update its nameservers when it starts and exits.

  Commands
  $ openvpn --script-security 2 --config cc00-myvpn.com_tcp.ovpn
  • Or 
  $ sudo openvpn --config cc00-myvpn.com_tcp.ovpn --up /etc/openvpn/update-resolv-conf --down /etc/openvpn/update-resolv-conf --script-security 2 --auth-user-pass /home/user/auth

Creating a autologin file

  • Configuring auth manually 
  $ sudo touch /home/user/auth 
  $ sudo nano /home/user/auth user password
  • A little protection 
  $ sudo chmod 600 /home/user/auth

OpenVPN Random Server Selection and Autologin

👷🛠️ UNDER CONSTRUCTION 🚧🏗

https://openvpn.net/community-resources/how-to/#auth

You could use the client.conf example below to random access multiple opvn files and auto login with auth configuration. Make the configurations refer to auth file by appending some directives at the end of each. Also create keepalive, a log record to facilitate troubleshooting and automatically run a script called update-resolv-conf, which may be necessary for DNS resolution to work correctly when enabling VPN and turn off. On Debian, this script is included with the OpenVPN installation.

  • Configuring client.conf manually
  $ sudo nano /etc/openvpn/client/client.conf
  client
  dev tun
  # It's TCP or UDP server?
  proto tcp
  remote my-server-1.com 1194
  remote my-server-2.com 1194
  remote my-server-3.com 1194
  remote my-server-4.com 1194
  remote my-server-5.com 1194
  remote my-server-6.com 1194
  remote my-server-7.com 1194
  remote my-server-8.com 1194
  remote my-server-9.com 1194
  remote my-server-10.com 1194
  remote-random # It chooses a random config server
  resolv-retry infinite
  nobind
  tun-mtu 1500
  tun-mtu-extra 32
  mssfix 1450
  persist-key
  persist-tun
  ping 15
  ping-restart 0
  ping-timer-rem
  reneg-sec 0
  comp-lzo no # Enable it if enabled in the server
  verify-x509-name CN=my.vpn-1.com # Protect against MITM
  remote-cert-tls server
  # Your autologin config
  auth-user-pass /etc/openvpn/client/auth
  # OpenVPN DNS Resolver
  script-security 2
  up /etc/openvpn/update-resolv-conf
  down /etc/openvpn/update-resolv-conf
  # Others
  keepalive 10 60
  log-append /var/log/openvpn.log
  verb 3
  pull
  fast-io
  cipher AES-256-CBC
  auth SHA512
  # Note SSL/TLS parms. See the server config file for more description.
  # It's best to use a separate .crt/.key file pair for each client.
  # A single ca file can be used for all clients.
  <ca>
  -----BEGIN CERTIFICATE-----
  -----END CERTIFICATE-----
  </ca>
  key-direction 1
  <tls-auth>
  # 2048 bit OpenVPN static key
  -----BEGIN OpenVPN Static key V1-----
  -----END OpenVPN Static key V1-----
  </tls-auth>

  • Configuring client.conf automatically
  $ sudo cd /etc/openvpn/client/
  $ sudo cat << EOF > client.conf
  client
  dev tun
  # It's TCP or UDP server?
  proto tcp
  remote my-server-1.com 1194
  remote my-server-2.com 1194
  remote my-server-3.com 1194
  remote my-server-4.com 1194
  remote my-server-5.com 1194
  remote my-server-6.com 1194
  remote my-server-7.com 1194
  remote my-server-8.com 1194
  remote my-server-9.com 1194
  remote my-server-10.com 1194
  remote-random # It chooses a random config server
  resolv-retry infinite
  nobind
  tun-mtu 1500
  tun-mtu-extra 32
  mssfix 1450
  persist-key
  persist-tun
  ping 15
  ping-restart 0
  ping-timer-rem
  reneg-sec 0
  comp-lzo no # Enable it if enabled in the server
  verify-x509-name CN=my.vpn-1.com # Protect against MITM
  remote-cert-tls server
  # Your autologin config
  auth-user-pass /etc/openvpn/client/auth
  # OpenVPN DNS Resolver
  script-security 2
  up /etc/openvpn/update-resolv-conf
  down /etc/openvpn/update-resolv-conf
  # Others
  keepalive 10 60
  log-append /var/log/openvpn.log
  verb 3
  pull
  fast-io
  cipher AES-256-CBC
  auth SHA512
  # Note SSL/TLS parms. See the server config file for more description.
  # It's best to use a separate .crt/.key file pair for each client.
  # A single ca file can be used for all clients.
  <ca>
  -----BEGIN CERTIFICATE-----
  -----END CERTIFICATE-----
  </ca>
  key-direction 1
  <tls-auth>
  # 2048 bit OpenVPN static key
  -----BEGIN OpenVPN Static key V1-----
  -----END OpenVPN Static key V1-----
  </tls-auth>
  EOF

  • Configuring client.conf automatically in batch
  $ echo 'auth-user-pass /etc/openvpn/client/auth
  keepalive 10 60
  log-append /var/log/openvpn.log
  script-security 2
  up /etc/openvpn/update-resolv-conf
  down /etc/openvpn/update-resolv-conf' | tee -a openvpn/*.conf

Create an autologin file

  • Configuring auth manually
  $ sudo touch /etc/openvpn/client/auth
  $ sudo nano /etc/openvpn/client/auth
  user
  password

  • Configuring auth automatically
  • If you are not going to copy the example, to create a newline (press ENTER) after you type the \
  to tell the shell you want to enter more parameters but on a separate line.
  $ cd /etc/openvpn/client/auth
  $ sudo echo 'user password' > openvpn/auth
  • A little protection
  $ chmod 600 /etc/openvpn/client/auth

  • Load daemon
  $ sudo openvpn --config /etc/openvpn/client.conf --daemon

  • Alternatively
  $ sudo chmod 600 /etc/openvpn/client/auth
  $ echo "USERNAME" | sudo tee -a /etc/openvpn/client/auth
  $ echo "PASSWORD" | sudo tee -a /etc/openvpn/client/auth
  $ chmod 600 /etc/openvpn/client/auth

  • Load daemon
  $ sudo openvpn --config /etc/openvpn/client.conf --daemon

  • Alternatively
  $ sudo su
  $ echo 'myuser' | sudo tee -a /etc/openvpn/client/auth
  $ echo 'mypassword' | sudo tee -a /etc/openvpn/client/auth
  $ chmod 600 /etc/openvpn/client/auth

  • Load daemon
  $ sudo openvpn --config /etc/openvpn/client.conf --daemon

  • Alternatively
  $ sudo su
  $ echo 'myuser' | tee --append /etc/openvpn/client/auth
  $ echo 'mypassword' | tee --append /etc/openvpn/client/auth
  $ chmod 600 /etc/openvpn/client/auth

  • Load daemon
  $ sudo openvpn --config /etc/openvpn/client.conf --daemon

OpenVPN DNS Resolver

👷🛠️UNDER CONSTRUCTION🚧🏗

https://wiki.archlinux.org/title/OpenVPN#DNS
https://github.com/jonathanio/update-systemd-resolved

"By default, all configured VPNs in /etc/openvpn/ are started during system boot. Edit /etc/default/openvpn to start specific VPNs or to disable this behavior. You need to run systemctl daemon-reload once to enable new VPNs."

  Commands 
  $ sudo su # cd /etc/openvpn/client # echo "script-security 2" | sudo tee -a /etc/openvpn/client/openvpn.conf # echo "up /etc/openvpn/update-resolv-conf" | sudo tee -a /etc/openvpn/client/openvpn.conf # echo "down /etc/openvpn/update-resolv-conf" | sudo tee -a /etc/openvpn/client/openvpn.conf

Solving DNS problems with OpenVPN

https://openvpn.net/vpn-server-resources/troubleshooting-dns-resolution-problems

OpenVPN DNS

👷🛠️UNDER CONSTRUCTION🚧🏗

$ sudo apt install resolvconf

Copy

*Consider

$ sudo apt install openvpn-systemd-resolved

Copy

$ sudo nano /etc/openvpn/update-resolv-conf

  $ sudo mv /etc/resolv.conf /etc/resolv.conf.bak
  • Add this lines into your openvpn client.conf:
  $ nano client.conf script-security 2 up /etc/openvpn/update-resolv-conf down /etc/openvpn/update-resolv-conf

Your could run openvpn with DNS resolver

  $ openvpn --script-security 2 --config cc00-myvpn.com_tcp.ovpn

Disabling OpenVPN's client DNS

$ sudo nano /etc/openvpn/client/client.conf

#Actual DNS name dhcp-option DNS 10.10.10.10

Take care with DNS leaks

curl ipleak.net/json/

curl ipinfo.io

#IPV4 pull-filter ignore "dhcp-option DNS" #IPV6 pull-filter ignore "dhcp-option DNS6"

Disabling NetworkManager's own dnsmasq

👷🛠️UNDER CONSTRUCTION🚧🏗

  $ sudo nano /etc/NetworkManager/NetworkManager.conf #dns=dnsmasq 
  $ sudo restart network-manager

NetworkManager dnsmasq (CLI)

DNS requests are directed to VPN-supplied DNS servers without any manipulations with dnsmasq, up/down/dispatch helper scripts.

mcli -p connection modify MY_VPN_CONNECTION ipv4.never-default no nmcli -p connection modify MY_VPN_CONNECTION ipv4.ignore-auto-dns no nmcli -p connection modify MY_VPN_CONNECTION ipv4.dns-priority -42

*Using OpenVPN through NetworkManager (GUI) allows users to disable the connection.

Enable OpenVPN as service at boot

To make OpenVPN automatically connect with a certain configuration, set the AUTOSTART directive in /etc/default/openvpn to the configuration filename without the extension.

  Commands
  • At boot, by default client.conf is enabled 
  $ sudo ls /etc/openvpn/client
  • Set the audoestart directive 
  $ sudo nano in /etc/default/openvpn AUTOSTART="nameofopvnconfigfile"
  • Save or edit your configuration with 
  $ sudo nano /etc/openvpn/client/[country].conf
  • Alternatively 
  $ sudo echo 'AUTOSTART="nameofopvnconfigfile"' | sudo tee -a /etc/default/openvpn
  • Enable the service by calling 
  $ sudo systemctl start openvpn-client@nameofopvnconfigfile 
  $ sudo systemctl enable openvpn-client@nameofopvnconfigfile
  • Verify 
  $ sudo cat /etc/default/openvpn
  • Load OpenVPN and connect 
  $ sudo systemctl daemon-reload 
  $ sudo systemctl restart openvpn

Test if the killswitch is working

  $ sudo systemctl start openvpn-client@[country] 
  $ sudo systemctl stop openvpn-client@[country] 
  $ sudo systemctl status openvpn-client@[country]
  $ curl ipleak.net/json/
  $ curl ipinfo.io

OpenVPN UFW kill switch

👷🛠️UNDER CONSTRUCTION🚧🏗

Set up a firewall to deny everything but the VPN handshake on the regular interfaces eth0 and wlan0 while placing no restrictions on tun0.

  $ sudo su # apt install ufw # ufw allow in on tun0 # ufw allow out on tun0 # ufw allow out on eth0 from any to any port 53 # ufw allow out on wlan0 from any to any port 53 # ufw allow out on eth0 from any to any port 1198 # ufw allow out on wlan0 from any to any port 1198 # ufw deny in on eth0 # ufw deny in on wlan0 # ufw deny out on eth0 # ufw deny out on wlan0 # ufw enable

Testing killswitch

  $ sudo systemctl start openvpn-client@[country] 
  $ sudo systemctl stop openvpn-client@[country] 
  $ sudo systemctl status openvpn-client@[country]
  $ curl ipleak.net/json/
  $ curl ipinfo.io

  $ systemctl stop openvpn
  $ curl --connect-timeout 5 ipinfo.io

OpenVPN DNS

👷🛠️UNDER CONSTRUCTION🚧🏗

resolv-conf

$ sudo apt install resolvconf

"Parses DHCP options from openvpn to update resolv.conf . To use set as 'up' and 'down' script in your openvpn *.conf:

up /etc/openvpn/update-resolv-conf down /etc/openvpn/update-resolv-conf

"Example envs set from openvpn:"

foreign_option_1='dhcp-option DNS 193.43.27.132' foreign_option_2='dhcp-option DNS 193.43.27.133' foreign_option_3='dhcp-option DOMAIN be.bnc.ch'

openvpn-systemd-resolved

$ sudo apt install openvpn-systemd-resolved

Copy

"OpenVPN helper to add DHCP information into systemd-resolved via DBus.(...)This script will parse DHCP options set via OpenVPN (dhcp-option) to update systemd-resolved directly via DBus, instead of updating /etc/resolv.conf. To install, set as the 'up' and 'down' script in your OpenVPN configuration file or via the command-line arguments, alongside setting the 'down-pre' option to run the 'down' script before the device is closed. For example:"

  up /etc/openvpn/scripts/update-systemd-resolved down /etc/openvpn/scripts/update-systemd-resolved down-pre

5.07 WireGuard

∙ WireGuard

https://github.com/WireGuard
https://wiki.ubuntuusers.de/WireGuard

$ sudo apt install wireguard

Copy

$ sudo apt install wireguard-tools

Copy

5.08 strongSwan

strongSwan

https://github.com/strongswan/strongswan
https://docs.strongswan.org/docs/5.9/index.html
https://docs.strongswan.org/docs/5.9/config/IKEv2.html
https://docs.strongswan.org/docs/5.9/config/logging.html

strongSwan (Client Side)

$ sudo apt install strongswan-charon

Copy

$ sudo apt install libcharon-extra-plugins

Copy

$ sudo apt install libcharon-extauth-plugins

Copy


  • Add username and password 
  $ sudo nano /etc/ipsec.secrets Username : EAP "Password" 
  • Configure 
  $ sudo su Password


  sudo printf '%s\n\t' 'conn MyVPN' \
  'keyexchange=ikev2' 'dpdaction=clear' 'dpddelay=300s' \
  'eap_identity="USERNAME"' 'leftauth=eap-mschapv2' 'left=%defaultroute' \
  'leftsourceip=%config' 'right=SERVER_IP' 'rightauth=pubkey' \
  'rightsubnet=0.0.0.0/0' 'rightid=%SERVER_HOSTNAME' \
  'rightca=/etc/ipsec.d/cacerts/VPN.pem' 'type=tunnel' 'auto=add' > /etc/ipsec.conf


  • Inside the file change load = yes to load = no. 
  $ sudo nano /etc/strongswan.d/charon/constraints.conf 
  • Download the VPN certificate. 
  $ sudo wget https://download/certificate/root.pem -O /etc/ipsec.d/cacerts/VPN.pem
  • Connecting 
  $ sudo ipsec restart 
  $ sudo ipsec up VPN “Connection VPN has been established successfully”. 
  • Disconnecting 
  $ sudo ipsec down VPN 
  • Debuging 
  $ sudo cat /var/log/syslog
  $ sudo ls /etc/strongswan.d/charon/

5.09 Firewall + VPN

∙ Firewall (UFW) + VPN (OpenVPN)

  Commands to setup UFW + OpenVPN
  • You could add specifically rules for each port separately on tun0 (VPN tunnel interface) 
  $ sudo ufw allow in on tun0 to any port 60000 proto tcp 
  $ sudo ufw allow in on tun0 to any port 60000 proto udp
  • You could test to connect in and out to anywhere on tun0 
  $ sudo ufw allow in on tun0 
  $ sudo ufw allow out on tun0
  • To allow access only from a specific address you could use 
  $ sudo ufw allow in on tun0 from 192.168.0.1 to any port 60000 proto tcp
  • Allow OpenVPN to connect to the regular network interface (e.g. eth0, wlan0...) through the ports present in the .opvn file (e.g.DNS resolution on port 53 and VPN server on 1198...) 
  $ sudo ufw allow out on eth0 from any to any port 53,1198
  • Consider this tcp or udp rules 
  $ sudo ufw allow out on eth0 to any port 53,1197 proto tcp 
  $ sudo ufw allow out on eth0 to any port 53,1197 proto udp
  • For a hard policy, working only with tun0, you could block the rest and enable the firewall 
  $ sudo ufw deny in on eth0 
  $ sudo ufw deny out on eth0
  • For a hard policy, you could block the rest and enable the firewall 
  $ sudo ufw status numbered 
  $ sudo ufw enable 
  $ sudo ufw reload 
  $ sudo reboot

  Commands to secure the server with iptables
  • Allow everything from within your VPN 
  $ sudo iptables -I INPUT -i tun0 -j ACCEPT
  • Explicitly allow what can be accessed within the VPN, for example, allow DNS and HTTP 
  $ sudo iptables -A INPUT -i tun0 -p tcp --destination-port 53 -j ACCEPT 
  $ sudo iptables -A INPUT -i tun0 -p udp --destination-port 53 -j ACCEPT 
  $ sudo iptables -A INPUT -i tun0 -p tcp --destination-port 80 -j ACCEPT
  • To enable SSH and VPN access from anywhere. 
  $ sudo iptables -A INPUT -p tcp --destination-port 22 -j ACCEPT 
  $ sudo iptables -A INPUT -p tcp --destination-port 1194 -j ACCEPT 
  $ sudo iptables -A INPUT -p udp --destination-port 1194 -j ACCEPT
  • To explicitly allow TCP/IP to do "three-way handshakes" 
  $ sudo iptables -I INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
  • To allow any loopback traffic, the server is allowed to talk to itself 
  $ sudo iptables -I INPUT -i lo -j ACCEPT
  • To reject access from anywhere else 
  $ sudo iptables -P INPUT DROP
  • To list rules 
  $ sudo iptables -L --line-numbers

(*ip6tables)

Troubleshooting iptables

  $ sudo systemctl restart servicedaemon.service 
  $ sudo systemctl restart service.service 
  $ sudo iptables -S
  $ ping duckduckgo.com

5.10 Leak Test

Leak Test

DNSLeakTest.com (run the "Extended test")
IPLeak.net
Mullvad DNS Leak Test
Surfshark DNS Leak Test
BrowserLeaks IP Test
IPX.AC DNS Leak Test

You could test your current public IP address and compare that to the one from before with 'ipleak.net'. If they match, your VPN is not working correctly.

$ curl ipleak.net/json/

Copy

$ curl ipinfo.io

Copy

$ curl --connect-timeout 5 ipinfo.io

Copy

5.11 Spoofing

https://github.com/alobbs/macchanger https://github.com/refraction-networking/utls https://github.com/0xsirus/tirdad

• Address Spoof

  Commands for a random MAC address
  $ ip link 
  $ sudo ifconfig wlan0 down 
  $ sudo macchanger -r wlan0
  • Shows specified MAC Address of NIC 
  $ sudo macchanger -s wlan0 
  $ sudo ifconfig wlan0 up

• Opt-Out WLAN-SSID

∙ To opt-out of global maps (https://wigle.net), rename your network WiFi SSID to

<SSID>_optout_nomap

∙ To opt-out of Mozilla Location Services

Go to https://location.services.mozilla.com/optout

• Hardware ID Spoofing

∙ BIOS Serial Number Spoofing

  # Install dmidecode to read hardware information
  $ sudo apt-get install dmidecode
  
  # Check current BIOS serial number
  $ sudo dmidecode -s system-serial-number
  
  # Spoof BIOS serial number (note: this requires BIOS-level tools, vendor-specific)
  # Refer to your motherboard's manual or vendor's tools for changing BIOS settings

∙ Hard Drive Serial Number Spoofing

  # Install hdparm
  $ sudo apt-get install hdparm
  
  # Check current hard drive serial number
  $ sudo hdparm -i /dev/sdX | grep SerialNo
  
  # Spoof hard drive serial number (note: this requires specific hardware support)
  # Some tools like hdparm can change specific parameters if supported by the hardware
  # Otherwise, hardware-level solutions are required

∙ Network Interface Card (NIC) MAC Address Spoofing

  # For Wi-Fi NIC
  $ sudo ip link set wlan0 down
  $ sudo macchanger -r wlan0
  $ sudo ip link set wlan0 up
  
  # For Ethernet NIC
  $ sudo ip link set eth0 down
  $ sudo macchanger -r eth0
  $ sudo ip link set eth0 up

• Firmware-Level Spoofing

∙ Custom Firmware Installation

  # Installing custom firmware on your devices can allow more control over hardware IDs
  # Example: Installing custom router firmware like OpenWRT or DD-WRT
  # Refer to specific device manuals for custom firmware installation

∙ Modify Device Firmware

  # This is an advanced technique requiring firmware modification skills
  # Tools and techniques vary widely depending on the hardware
  # Refer to communities like XDA Developers for Android devices, router firmware forums, etc.

5.12 Others

• Torrenting

https://portforward.com https://wiki.wireshark.org/BitTorrent https://github.com/LiamTheBox/Torrent-With-A-VPN https://github.com/mdlam92/vpn_torrenting https://github.com/tool-maker/VPN_just_for_torrents/wiki https://askubuntu.com/questions/559016/ufw-rules-dont-block-deluge https://transmissionbt.com https://comparitech.com/blog/vpn-privacy/how-to-make-a-vpn-kill-switch-in-linux-with-ufw

👷🛠️UNDER CONSTRUCTION🚧🏗

Transmission remotely access

  Commands for remote Transmission 
  $ sudo apt install transmission-cli 
  $ sudo apt install transmission-common 
  $ sudo apt install transmission-daemon 
  $ sudo service transmission-daemon stop
  • To 
  $ sudo nano /etc/transmission-daemon/settings.json > “rpc-whitelist”: “127.0.0.1,192.168.*.*”, > “rpc-whitelist-enabled”: true,
  • To change the download directory > "download-dir": /home/user/Downloads 
  $ sudo service transmission-daemon start
  • To find local IP address
  $ hostname -I
  • To find local MAC address 
  $ sudo cat /sys/class/net/eth0/address
  • In your browser > https://192.168.0.15:9091 > Login: transmission > Password: transmission

6. SOFTWARES

6.01 Office Productivity

01 Office Softwares

Word Completion for Text Documents

To Accept/Reject a Word Completion

By default, you accept the word completion by pressing the Enter key.
To reject the word completion, continue typing with any other key.

To Switch off the Word Completion

Choose Tools - AutoCorrect - AutoCorrect Options - Word Completion.
Uncheck Enable word completion.

Cursor Position

In general, all documents open with the cursor at the start of the document.

One exception appears when the author of a Writer text document saves and reopens a document: The cursor will be at the same position where it has been when the document was saved. This only works when the name of the author was entered in Tools - Options - LibreOffice - User Data .

Press Shift+F5 to set the cursor to the last saved position.

Version Management

The File menu contains a Versions command that enables you to save multiple versions of a document in the same file.

You can choose to view individual versions of a document, or you can display the differences between versions with color markings.

In the dialog to open a document, you can select from a combo box which version of this document you want to open.

Saving Documents Automatically

To create a backup file every time you save a document Choose Tools - Options - Load/Save - General.

Mark Always create backup copy.

If the Always create backup copy option is selected, the old version of the file is saved to the backup directory whenever you save the current version of the file.

You can change the backup directory by choosing Tools - Options - LibreOffice - Paths, then change the Backups path in the dialog.

The backup copy has the same name as the document, but the extension is .BAK. If the backup folder already contains such a file, it will be overwritten without warning.

To save recovery information automatically every n minutes Choose Tools - Options - Load/Save - General.

Mark Save AutoRecovery information every and select the time interval.

This command saves the information necessary to restore the current document in case of a crash. Additionally, in case of a crash LibreOffice tries automatically to save AutoRecovery information for all open documents, if possible.

Zotero

Zotero - Your personal research assistant.

Zotero
Zotero - Support
Zotero - Default Translators
Zotero - Item Types and Fields
Zotero - Citing Fields from "Extra" (Exporting - Extra fields)

Place (Publisher and Publisher Place)	`publisher: Publisher` `publisher-place: Publisher Place`	For Presentations, the place where the meeting was held or the presentation was made. For Conference Papers (published in a conference proceedings), use this field for the place where the proceedings was published. If separate locations are needed for the publication place and the location of the conference, leave this field blank and add Event Place and Publisher Place fields to Extra
Archive Place	`archive-place: Archive Place`	The geographic location of an archive.

Original Title	`original-title: Original Title`	The original title of a work (e.g., the untranslated title).
Original Publisher	`original-publisher: Original Publisher`	The publisher of the original version of an item (e.g., the untranslated version).
Original Publisher Place	`original-publisher-place: Original Publisher Place`	The geographic location of the publisher of the original version of an item (e.g., the untranslated version).

Issue Date, Date Decided or Enacted	`issued: Issue Date`	The original date an item was published. Enter in ISO format (year-month-day).
Submitted Date or Filing Date	`submitted: Submitted`	The date an item was submitted for publication.
Access Date	`Accessed`	Date an electronic resource was accessed.
Event Date	`event-date: Event Date`	The date an event took place. Enter in ISO format (year-month-day).
Original Date	`original-date: Original Date`	The original date an item was published. Enter in ISO format (year-month-day).

Zotero - Groups
Zotero - Importing standardized bib. formats

*Importing bibliographic data: the most popular formats are BibLaTex (.bib), RIS (.ris) and MODS (.xml).

Juris-M (For Legal Citations)

Jurism is based on Zotero reference manager, to which it adds feature for handling legal and multilingual resources.

Juris-M for heavy or frequent legal citations for US, UK and GE legal cases and legislation.

It is possible to create proper citation for basic legal citations in Zotero, particularly if only a few such citations are needed.

Juris-M
Zotero - Legal Citations: Juris-M

Basic Text Editors

Comparison of Basic Text Editors

Aspect	CherryTree	Zim	Xournal++	RedNotebook	FeatherNotes
Type	Hierarchical Note-taking	Desktop Wiki-style Note-taking	Note-taking and PDF Annotation	Diary and Journal	Note-taking
Rich Text Editing	Yes	Yes	Yes (handwritten notes)	Yes	Yes
Hierarchical Structure	Yes	Yes	No	No	No
Encryption	Yes	No	No	No	No
Image Support	Yes	Yes	Yes	Yes	Yes
Export Options	HTML, PDF	HTML, LaTeX	PDF	HTML, PDF	HTML
Custom Shortcuts	Yes	Yes	Limited	Limited	No
Dark Mode	Yes	Yes	Yes	Yes	Yes
Code Formatting (Syntax Highlighting)	Yes	Yes	No	Limited	Yes
Tagging	Yes	Yes	No	Yes	No
Support for Markdown	Yes	Yes	No	Yes	No
Storage Method	Single XML or SQLite file	Plain text files	Custom .xopp files	Plain text files	Plain text files
Cross-Platform	Yes (Windows, macOS, Linux)	Yes (Windows, macOS, Linux)	Yes (Windows, macOS, Linux)	Yes (Windows, macOS, Linux)	Yes (Windows, macOS, Linux)
Command to Install on Debian	`sudo apt install cherrytree`	`sudo apt install zim`	`sudo apt install xournalpp`	`sudo apt install rednotebook`	`sudo apt install feathernotes`

02 Password Managers

Password Manager

• KeePassXC

https://keepassxc.org/docs/ https://cvedetails.com/vendor/12214/Keepass.html

$ sudo apt install keepassxc

Copy

03 Email Clients

Email

Thunderbird

https://thunderbird.net

$ sudo apt install -y thunderbird

Copy

$ sudo apt install -y birdtray

Copy

NeoMutt

https://neomutt.org

NeoMutt is a command line mail reader based on Mutt.

$ sudo apt install -y neomutt

Copy

Encrypted Emails

https://emailselfdefense.fsf.org/en/
https://emailselfdefense.fsf.org/en/workshops.html
https://riseup.net/en/security/message-security/openpgp/best-practices
https://riseup.net/en/security/message-security/openpgp/enigmail
https://linuxbabe.com/security/encrypt-emails-gpg-thunderbird
https://wiki.archlinux.org/title/Paperkey
https://keys.openpgp.org/about/usage
https://efail.de

Note 1: You cannot recover the secret key from the public key and the passphrase. You cannot recover your secret gpg key without a backup.

Note 2: Create an expiration date for security reasons.

👷🛠️UNDER CONSTRUCTION🚧🏗

Note 3: Create an .

  Commands for gnupg (GnuPG - GNU Privacy Guard)
  • How to export and import GPG key:
  $ gpg --export ${ID} > public.key
  $ gpg --export-secret-key ${ID} > private.key
  $ gpg --import --batch public.key
  $ gpg --import --batch backup_dir/.gnupg/pubring.gpg
  $ gpg --import --batch backup_dir/.gnupg/secring.gpg
  $ gpg --edit-key ${KEY} trust quit
  $ gpg --list-keys
  $ gpg --list-secret-keys

  Commands for gnupg (GnuPG - GNU Privacy Guard)
  • How to extend the expiration date of an already expired GPG key:
  $ gpg --list-keys
  $ gpg --edit-key (key id)
  • GPG console will open in the primary key, select a sub-key: gpg> gpg> list gpg> key 1
  • Set the expiration for the selected key gpg> expire gpg> save
  • After update, you can send it out gpg --keyserver site.com --send-keys (key id)

gpg --list-secret-keys --verbose --with-subkey-fingerprints

04 Office Utilities

Office Utilities

$ sudo apt install xpad

Copy

$ sudo apt install kcalc

Copy

05 Display Tools

Utilities

Set color temperature of display

Redshift

$ sudo apt install redshift

$ sudo apt install redshift-gtk

redshift.conf

https://github.com/raw/jonls/redshift/master/redshift.conf.sample

$ ~/.config/redshift/redshift.conf

Copy

$ redshift -P -O TEMPERATURE

Copy

$ redshift -P -O 4000

Copy

$ redshift -P -O 6000

Copy

$ sudo apt install brightnessctl

Copy

$ brightnessctl s 25% && redshift -P -O 4000

Copy

$ brightnessctl s 50% && redshift -P -O 6500

Copy

$ redshift -l LAT:LONG

Copy

6.02 Document Management

01 PDF Suites

PDF Reader

Qpdf

$ sudo apt install -y qpdf

Copy

https://cvedetails.com/vendor/16505/Qpdf-Project.html

Okular

$ sudo apt install -y okular

Copy

$ sudo apt install -y okular-extra-backends

Copy

okular-extra-backends: additional document format support for Okular.These plugins allow Okular to view additional document formats:

EPUB
Markdown
DeJaVu Format
TIFF Image Format
Microsoft CHM Format

PDF Editor

PDF Arranger (GUI)

https://github.com/pdfarranger/pdfarranger

$ sudo apt install -y pdfarranger

Copy

Manually Crop or Remove Pages: This tool is more for rearranging and cropping pages, but it can also be useful for removing unwanted sections.

How to combine PDFs in CLI

$ sudo apt install -y ghostscript

  • Command to combine
  $ gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=combined.pdf file1.pdf file2.pdf
  • Output in low resolution
  $ gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress -sOutputFile=merged.pdf mine1.pdf mine2.pdf

PDF Crop and Split

Krop (GUI)

https://arminstraub.com/software/krop

Krop is designed to adjust which parts of a PDF are displayed by changing the crop box of the PDF. This means the original content is still in the file and can be revealed or accessed using a PDF editor or viewer that ignores the crop box. Krop is not suited for censoring a PDF document or decreasing the size of a PDF file.

$ sudo apt install -y krop

Copy

  • To automatically undo 4 pages print onto a single page:
  $ krop --go --grid=2x2 input.pdf
  • To trim each of these pages:
  $ krop --go --grid=2x2 --trim --trim-use=all input.pdf
  • Others krop --grid=2x1 --initialpage=3 --exceptions=1 --trim-use=all --trim ~/input.pdf

--trim: Trims whitespace around the content. --trim-use=all: Uses all pages to determine the trim area, ensuring a consistent crop.

pdfcrop (texlive-extra-utils)

$ sudo apt install -y texlive-extra-utils

Adjust the margins, effectively trimming the PDF. pdfcrop is a tool that uses Ghostscript to crop whitespace or specified margins from a PDF.

  • Command
  $ pdfcrop input.pdf output.pdf
  $ pdfcrop --margins '-10 -10 -10 -10' input.pdf output.pdf
  $ pdfcrop --margins '0 0 0 -50' input.pdf output.pdf

pdftk

$ sudo apt install -y pdftk

  • Command to remove page 4
  $ pdftk input.pdf cat 1-3 5-end output output.pdf
  • Command 
  $ pdftk input.pdf cat 1-endeast output rotated.pdf

pdfjam

$ sudo apt install -y pdfjam

  • Commands
  $ pdfjam --twoside --offset '0cm 0cm' --paper a4paper cropped.pdf --outfile final.pdf

02 Image Convert

Image Convert

Convert image with ImageMagick

https://imagemagick.org/script/formats.php
https://imagemagick.org/script/mogrify.php
https://imagemagick.org/script/command-line-tools.php
https://cvedetails.com/vendor/1749/Imagemagick.html

$ sudo apt install imagemagick

Copy

*Note that mogrify is part of ImageMagick package. The command convert or magick convert is an old "legacy" program, provided for compatabilty only, don't use it for new work. Instead, use just magick mogrify or just mogrify .

  • One-liner commands
   $ convert input.svg output.png
   $ mogrify -format png *.svg
   $ convert input.jpg output.png
   $ mogrify -format png *.jpg
   $ mogrify -format png *.jpeg
   $ mogrify -format png *.webp
   $ mogrify -format png *.avif
   $ mogrify -format png *.gif
  • Batch processing commands
   $ cd ~/Donwloads
   $ find . -name "*.jpg" -exec mogrify -monitor -format png {} \;
   $ find . -name "*.jpeg" -exec mogrify -monitor -format png {} ;
   $ find . -name "*.webp" -exec mogrify -monitor -format png {} \;
   $ find . -name "*.svg" -exec mogrify -monitor -format png {} \;
   $ find . -name "*.avif" -exec mogrify -monitor -format png {} \;
   $ find . -name "*.gif" -exec mogrify -monitor -format png {} \;

Rotate image with ImageMagick

UNROTATE UNROTATE2 UNROTATE3

$ sudo apt install imagemagick

Copy

  • Commands
  $ mogrify -monitor -rotate -90 *.png

Deskew text-image with ImageMagick

TEXTDESKEW

  • Commands
  $ magick [image_source]  -set option:angle "%[minimum-bounding-box:unrotate]" -background white -rotate "%[angle]" [image_target]
  $ magick input.jpeg -deskew 60% output.jpg

Unpaper

Unpaper internal - OCRFeeder (GUI)

https://wiki.gnome.org/Apps/OCRFeeder

$ sudo apt install -y ocrfeeder

Copy

Tools -> Unpaper

Unpaper internal - OCRmyPDF (CLI)

https://ocrmypdf.readthedocs.io
https://ocrmypdf.readthedocs.io/en/latest/cookbook.html

$ sudo apt install -y ocrmypdf

Copy

$ ocrmypdf --clean

Copy

$ ocrmypdf --clean-final

Copy

$ ocrmypdf --remove-background

Copy

Note that --clean-final and --remove-background may leave undesirable visual artifacts in some images where their algorithms have shortcomings. Files should be visually reviewed after using these options.

--remove-background attempts to detect and remove a noisy background from grayscale or color images. Monochrome images are ignored. This should not be used on documents that contain color photos as it may remove them.

--clean uses unpaper to clean up pages before OCR, but does not alter the final output. This makes it less likely that OCR will try to find text in background noise.

--clean-final uses unpaper to clean up pages before OCR and inserts the page into the final output. You will want to review each page to ensure that unpaper did not remove something important.

--clean uses unpaper to clean up pages before OCR, but does not alter the final output. This makes it less likely that OCR will try to find text in background noise.

Convert with webp (dwebp)

$ sudo apt install webp

Copy

  Commands for webp files 
  • How to convert .webp to .png #It's a command-line interface
  $ dwebp -v input.webp -o ~/output.png
  $ dwebp -v -resize width x height input.webp -o ~/output.png *If either (but not both) of the width or height parameters is 0, the value will be calculated preserving the aspect-ratio.

  Commands for webp files in batch 
  $ for file in *.webp ; do dwebp "$file" -o "${file%.*}.png" ; done 
  • Testing alternatives 
  $ find . -name "*.webp" -exec dwebp {} -o "${file%.*}.png" \; 
  $ find . -type f -name "*.webp" -exec dwebp {} -o *.png 
  $ sudo apt install parallel 
  $ parallel dwebp {} -o *.png 
  $ find . -name "*.webp" -print0 | parallel --progress -0 dwebp {} -o *.png
   $ for x in 'ls -1 *.jpg'; do dwebp {} -o ${x%.*}.png ; done 
  $ for x in 'find . -name "*.webp"'; do dwebp {} -o ${x%.*}.png ; done

03 PDF Convert

Libre Office (Headless)

See LibreOffice documentation for more information.

$  sudo apt install -y libreoffice

Commands for LibreOffice Headless

Syntax:

$  soffice --convert-to OutputFileExtension[:OutputFilterName[:OutputFilterParams[,param]]] [--outdir output_dir]

To convert a DOCX file to PDF:

$  soffice --headless --convert-to pdf:writer_pdf_Export --outdir /home/user *.docx

To convert a ODT file to PDF:

$  soffice --headless --convert-to pdf:writer_pdf_Export --outdir /home/user *.odt

Output as PDF

To control which LibreOffice component generates PDF output, you can use these variants:

  --convert-to pdf:writer_pdf_Export 
  --convert-to pdf:calc_pdf_Export 
  --convert-to pdf:draw_pdf_Export 
  --convert-to pdf:impress_pdf_Export 
  --convert-to pdf:writer_web_pdf_Export

Ghostscript

https://ghostscript.com
https://cvedetails.com/vendor/10846/Artifex.html

$ sudo apt install -y ghostscript

Commands to convert PDF

$ gs -sDEVICE=pngalpha -o output.png -r144 input.pdf

ImageMagick (GUI or CLI)

Imagemagick

Commands to convert PDF

$ convert input.pdf output.png

Pandoc

Pandoc - a universal document converter https://pandoc.org/.

$ sudo apt install -y pandoc

Commands for Pandoc

Convert ODT to DOCX: $ pandoc -o document.odt document.docx
Convert DOCX to PDF: $ pandoc -s document.docx -o document.pdf

04 PDF Optimizers

ImageMagick (GUI or CLI)

https://imagemagick.org/Usage/crop
https://imagemagick.org/Usage/crop/#crop_repage
https://cvedetails.com/vendor/1749/Imagemagick.html

$ sudo apt install imagemagick

Copy

*Note that mogrify is part of ImageMagick package. The command convert or magick convert is an old "legacy" program, provided for compatabilty only, don't use it for new work. Instead, use just magick mogrify or just mogrify.

Commands to crop .pdf

$ mogrify -monitor 'ls input-*.png' -crop 3704x1852+160+20 output.png

$ mogrify -monitor -crop 1000x1350+20+145 +repage -path cropped *.png

Monitor progress: -monitor

Print detailed information about the image: -verbose

Commands to reduce .pdf size

$ mogrify -monitor -density 300x300 -quality 100 input.pdf output.pdf

$ mogrify -monitor -density 200x200 -quality 60 -compress jpeg input.pdf output.pdf

$ mogrify -monitor -density 150x150 -quality 70 -compress jpeg -resize 15% input.pdf output.pdf

$ mogrify -monitor -density 150x150 -compress Zip input.pdf output.pdf

$ mogrify -monitor -density 80 -page a4 input.pdf output.pdf

$ mogrify -monitor input.pdf -resample 85% output.pdf

$ mogrify -monitor *.png -colorspace gray -resample 100% "input.pdf"

• Commands to scanned books

$ mogrify -normalize -density 300 -depth 8 *.png

$ mogrify -normalize -density 300 -depth 8 -crop 50%x100% +repage *.png

$ mogrify -monochrome -normalize -density 300 *.png

-normalize: increase the contrast in an image by stretching the range of intensity values.

-depth: the number of bits per channel for each pixel.

-monochrome: transform the image to black and white.

pdfCropMargins - Python

https://pypi.org/project/pdfCropMargins

  $ pip install "pdfCropMargins" --upgrade
  $ pdf-crop-margins -v -p 0 -a -6 input.pdf

Ghostscript

https://ghostscript.com
https://cvedetails.com/vendor/10846/Artifex.html

$ sudo apt install -y ghostscript

Commands to optimize PDF

Rewrites the PDF, sometimes cleaning up unwanted elements.

  $ gs -o output.pdf -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress input.pdf

Convert PDF - Reduce size of scanned book

  $ gs -dNOPAUSE -dBATCH -dQUIET \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/screen \
  -sOutputFile=output.pdf \
  input.pdf

  $ gs -dNOPAUSE -dBATCH -dQUIET \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/printer \
  -sOutputFile=output.pdf \
   input.pdf

  $ gs -dNOPAUSE -dBATCH -dQUIET \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/prepress \
  -dDetectDuplicateImages \
  -dCompressFonts=true -r150 \
  -sOutputFile=output.pdf \
   input.pdf

  $ gs -dNOPAUSE -dBATCH -dQUIET \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/prepress \
  -dDetectDuplicateImages \
  -dCompressFonts=true -r300 \
  -sOutputFile=output.pdf \
   input.pdf

Convert PDF - Reduce size of Acrobat PDF

*Test results with comp. 1.3 and 1.4 .

  $ gs -dNOPAUSE -dBATCH -dSAFER \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/ebook \
  -dEmbedAllFonts=true \
  -dSubsetFonts=true \
  -dColorImageDownsampleType=/Bicubic \
  -dColorImageResolution=96 \
  -dGrayImageDownsampleType=/Bicubic \
  -dGrayImageResolution=96 \
  -dMonoImageDownsampleType=/Bicubic \
  -dMonoImageResolution=96 \
  -sOutputFile=output.pdf input.pdf

  $ gs -dNOPAUSE -dBATCH -dSAFER \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen \
  -dEmbedAllFonts=true \
  -dSubsetFonts=true \
  -dColorImageDownsampleType=/Bicubic \
  -dColorImageResolution=144 \
  -dGrayImageDownsampleType=/Bicubic \
  -dGrayImageResolution=144 -dMonoImageDownsampleType=/Bicubic \
  -dMonoImageResolution=144 \
  -sOutputFile=output.pdf input.pdf

  • Batch combine tips 
  $ cd /folder 
  $ find . -name '*.pdf' -exec gs -o -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress -sOutputFile=../output.pdf {} +

  • Batch process all pdfs one by one renaming the output 
  $ cd /folder 
  $ find . -name '*.pdf' -exec sh -c ' for pdf; do output="${pdf%.pdf}-processed.pdf" gs -dNOPAUSE -dBATCH -dSAFER \
  -sDEVICE=pdfwrite \
  -dCompatibilityLevel=1.4 \
  -dPDFSETTINGS=/screen \
  -dEmbedAllFonts=true \
  -dSubsetFonts=true \
  -dColorImageDownsampleType=/Bicubic \
  -dColorImageResolution=144 \
  -dGrayImageDownsampleType=/Bicubic \
  -dGrayImageResolution=144 -dMonoImageDownsampleType=/Bicubic \
  -dMonoImageResolution=144 \
  -sOutputFile="$output" "$pdf" done ' sh {} +

  • Name tips 
  -sOutputFile=ABC-%d.png produces 'ABC-1.png', ... , 'ABC-10.png', .. 
  -sOutputFile=ABC-%03d.pgm produces 'ABC-001.pgm', ... , 'ABC-010.pgm', ... 
  -sOutputFile=ABC_p%04d.tiff produces 'ABC_p0001.tiff', ... , 'ABC_p0510.tiff', ... , 'ABC_p5238.tiff'

• References -dPDFSETTINGS=/screen - Low quality and small size at 72dpi. -dPDFSETTINGS=/ebook - Slightly better quality but also a larger file size at 150dpi. -dPDFSETTINGS=/prepress - High quality and large size at 300 dpi. -dPDFSETTINGS=/default - System chooses the best output, which can create larger PDF files.

  Commands for ebook-convert
  • How to convert .epub to .pdf 
  $ sudo apt install calibre
  $ ebook-convert input  .epub output.pdf
  $ ebook-convert input.epub output.pdf --enable-heuristics
  $ find ./ -iname "*pdf" -type f | while read f; do echo -e "\e[1mConverting file $f \e[0m" ; ebook-convert "$f" "${f%.pdf}.epub" --enable-heuristics ; done

*Ref.: https://manpages.debian.org/bookworm/calibre/ebook-convert.1.en.html

*Utility.: https://convertfiles.com

  Commands for ps2pdf
  • How to convert .ps to .pdf 
  $ sudo apt install ps2pdf
  $ ps2pdf -dPDFSETTINGS=/ebook input.pdf output.pdf

*LibreOffice Draw: DPI of 100 and JPEG compression of 80%.

*Try:

  $ ps2pdf input.pdf output.pdf

05 PDF Crop

06 PDF OCR

• PDF OCR - Optical Character Recognition

OCRFeeder (GUI)

https://wiki.gnome.org/Apps/OCRFeeder

$ sudo apt install -y ocrfeeder

Copy *Unpaper

Cuneiform (CLI)

https://packages.debian.org/bookworm/cuneiform

OcrmOCRmyPDF (CLI)

https://ocrmypdf.readthedocs.io

$ sudo apt install -y ocrmypdf

Copy

Also install the Tesseract OCR plugins for your desired language

$ sudo apt install -y tesseract-ocr-eng

Copy

$ sudo apt install -y tesseract-ocr-deu

Copy

$ sudo apt install -y tesseract-ocr-fra

Copy

$ sudo apt install -y tesseract-ocr-spa

Copy

$ sudo apt install -y tesseract-ocr-por

Copy

$ sudo apt install -y tesseract-ocr-rus

Copy

$ sudo apt install -y tesseract-ocr-ara

Copy

$ sudo apt install -y tesseract-ocr-chi-sim

Copy

$ sudo apt install -y tesseract-ocr-chi-tra

Copy

OCRmyPDF Commands and Settings

Basic commands

How to OCR a PDF
```
$ ocrmypdf -v input.pdf output.pdf
```
$ ocrmypdf -v --language eng input.pdf output.pdf
$ ocrmypdf -v --language eng+deu input.pdf output.pdf
$ ocrmypdf -v --language eng+spa input.pdf output.pdf
$ ocrmypdf -v --language por+deu input.pdf output.pdf

To modify a file in the same place
$ ocrmypdf -v ~/input.pdf ~/input.pdf

To skip text
$ ocrmypdf -v --skip-text input.pdf output.pdf

To redo OCR
$ ocrmypdf -v --redo-ocr input.pdf output.pdf

Compression settings

$ ocrmypdf -v --pdfa-image-compression=jpeg --language=por+deu input.pdf output.pdf
$ ocrmypdf -v --pdfa-image-compression=lossless --language=por+deu input.pdf output.pdf
$ ocrmypdf -v --output-type=pdf --language por+deu input.pdf output.pdf

OcrmOCRmyPDF - Image processing

$ ocrmypdf -v --clean --language=por+deu input.pdf output.pdf
$ ocrmypdf -v --clean-final --language=por+deu input.pdf output.pdf
$ ocrmypdf -v --remove-background --language=por+deu input.pdf output.pdf

Warning

In many cases image processing will rasterize PDF pages as images, potentially losing quality. We caution against using ImageMagick or Ghostscript to convert images to PDF, since they may transcode images or produce downsampled images, sometimes without warning.

OCRmyPDF perform some image processing on each page of a PDF, if desired. The same processing is applied to each page. It is suggested that the user review files after image processing as these commands might remove desirable content, especially from poor quality scans.

Note that --clean-final and --remove-background may leave undesirable visual artifacts in some images where their algorithms have shortcomings. Files should be visually reviewed after using these options.

--clean uses unpaper to clean up pages before OCR, but does not alter the final output. This makes it less likely that OCR will try to find text in background noise.

--clean-final uses unpaper to clean up pages before OCR and inserts the page into the final output. You will want to review each page to ensure that unpaper did not remove something important.

--remove-background attempts to detect and remove a noisy background from grayscale or color images. Monochrome images are ignored. This should not be used on documents that contain color photos as it may remove them.

OcrmOCRmyPDF - PDF optimization

$ ocrmypdf -v --optimize=[0,1,2,3] input.pdf output.pdf

By default OCRmyPDF will attempt to perform lossless optimizations on the images inside PDFs after OCR is complete. Optimization is performed even if no OCR text is found.

The --optimize N (short form -O) argument controls optimization, where N ranges from 0 to 3 inclusive, analogous to the optimization levels in the GCC compiler.

Optimization is improved when a JBIG2 encoder is available and when pngquant is installed. If either of these components are missing, then some types of images cannot be optimized.

The types of optimization available may expand over time. By default, OCRmyPDF compresses data streams inside PDFs, and will change inefficient compression modes to more modern versions. A program like qpdf can be used to change encodings, e.g. to inspect the internals for a PDF.

ocrmypdf --optimize 3 in.pdf out.pdf # Make it small Some users may consider enabling lossy JBIG2. See: jbig2-lossy.

Image processing and PDF/A conversion can also introduce lossy transformations to your PDF images, even when --optimize 1 is in use.

OcrmOCRmyPDF - PDF Rotation

$ ocrmypdf -v --deskew input.pdf output.pdf
$ ocrmypdf -v --rotate-pages input.pdf output.pdf
$ ocrmypdf -v --rotate-pages-threshold {0.0-2.0} input.pdf ou
$ ocrmypdf -v --deskew --rotate-pages --clean input.pdf output.pdf
$ ocrmypdf -v --deskew --skip-text input.pdf output.pdf
$ ocrmypdf -v --deskew --tesseract-timeout=0 input.pdf output.pdf

--rotate-pages attempts to determine the correct orientation for each page and rotates the page if necessary.

--deskew will correct pages that were scanned at a skewed angle by rotating them back into place.

--clean uses unpaper to clean up pages before OCR, but does not alter the final output. This makes it less likely that OCR will try to find text in background noise.

--clean-final uses unpaper to clean up pages before OCR and inserts the page into the final output. You will want to review each page to ensure that unpaper did not remove something important.

08 PDF Others

PDF - Bookmarks Creation

https://github.com/SiddharthPant/booky

PDF - Remove Annotations

Removing annotations at once in Okular

View a page that has an annotation, find them in the annotation side pane. Right-click on the annotation icon in the document, and click Remove Annotation. Then save the changes to a new document by clicking the menu button in the top right, followed by Save As….

  Commands for pdftocairo
  $ pdftocairo -pdf "input.pdf" "output-with-flatten-annotations.pdf"

  Commands for qpdf
  $ qpdf --flatten-annotations=all input.pdf output.pdf

*May apply some differences.

*May result in larger PDF.

PDF - Remove Hiperlinks

PDFtk

$ sudo apt install pdftk

Copy

Remove all interactive elements, including hyperlinks.

  pdftk input.pdf output cleaned.pdf

qpdf

$ sudo apt install qpdf

Copy

  
  qpdf --linearize --replace-input input.pdf cleaned.pdf

PDF - Remove Metadata

Exiftool

$ sudo apt install exiftool

Copy

*Note: Exiftool PDF Tags "All metadata edits are reversible. While this would normally be considered an advantage, it is a potential security problem because old information is never actually deleted from the file. (However, after running ExifTool the old information may be removed permanently using the "qpdf" utility with this command: "qpdf --linearize in.pdf out.pdf".)"

*Note: PDF Sanitize

  qpdf --sanitize input.pdf cleaned.pdf
  qpdf --linearize --sanitize input.pdf output.pdf

--linearize: Does not remove any content but reorganizes it for optimized loading, focuses on optimizing the file for faster web viewing. Useful when hosting PDFs on websites to ensure users can start reading the document before the entire file is downloaded.

--sanitize: Removes specific types of content like JavaScript and embedded files, focuses on making the file safe by removing potentially harmful content. Useful when sharing PDFs to ensure they do not contain any malicious content that could harm the recipient's system.

Ghostscript

$ sudo apt install gs

Copy

  gs -dSAFER -dBATCH -dNOPAUSE -dNOCACHE -sDEVICE=pdfwrite -sOutputFile=cleaned.pdf -f input.pdf
  gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dPrinted=false -sOutputFile=output.pdf input.pdf

PDF - Read and Edit Metadata

pdfinfo

$ sudo apt install poppler-utils

Copy

Poppler-utils package contains pdfinfo.

$ pdfinfo input.pdf

Copy

$ pdfinfo -meta input.pdf

Copy

$ pdfinfo -js filename.pdf

Copy

Exiftool

$ sudo apt install exiftool

Copy

  Commands for exiftool basic commands 
  • Remove all metadata from all files possible inside a folder and all its ubfolders without backup (take care, might affect the colors) 
  $ exiftool -v -all:all= -overwrite_original -r /path 
  • Shows metadata: 
  $ exiftool -a -Title input.pdf 
  $ exiftool -G1 -a input.pdf 
  $ exiftool -G1 -s 'input.pdf' | grep '\[PDF\]' 
  $ exiftool -a -Model -ImageSize photo.jpg 
  • Process all files of specified file type (case insensitive extension) 
  $ exiftool -v -Model -ImageSize -ext jpg /path/to/files/ 
  • Recursively process all jpg files under specified directory and sub-directory 
  $ exiftool -v -r -Model -ImageSize -ext jpg /path/to/files/

  • Editing PDF metadata from command line 
  $ exiftool -Title=" " /
  -Author=" " /
  -Description=" " /
  -Subject=" " /
  -Date=" " /
  -CreationDate=" " /
  -Publisher=" " /
  -ISBN=" " /
  input.pdf

*To not create a backup in command-line the option is -overwrite_original.

*To not creat a backup in ExifToolGUI, there's menu "Options ">"Don 't backup files when modifying".

PFD - Remove Watermark

pdftoppm

$ sudo apt install pdftoppm

Copy

If the watermark is at a fixed position on each page, you can use pdftoppm command (part of Poppler utilities package) and convert command (from ImageMagick package) to remove the watermark by cropping the pages.

$ pdftoppm -png -cropbox input.pdf output

Copy

Ghostscript

$ sudo apt install gs

Copy


  gs -o output.pdf -sDEVICE=pdfwrite \
  -c "[/Page 1 /View [/CropBox [0 0 612 792]] /Rect [0 0 612 792] /Action << /Subtype /Hide /O 0>> /Subtype /Link /ANN pdfmark" \
  -f input.pdf

This command tries to create a link annotation that hides content on the first page. Adjust /Page 1 and coordinates [0 0 612 792] as necessary to target specific areas or pages.

6.03 Internet and Cloud

01 Browsers

Browser Comparison

Recent Browser Security Vulnerabilities

CVE Code	Description	Browser Affected	CVE Score
CVE-2024-29944	Privileged JavaScript Execution via Event Handlers	Mozilla Firefox	8.8
CVE-2023-7024	Use-after-free vulnerability in Blink rendering engine of Google Chrome	Google Chrome	9.8
CVE-2023-42917	Remote code execution in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-42916	Remote code execution in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-5217	Memory corruption in V8 JavaScript engine of Google Chrome	Google Chrome	9.8
CVE-2023-41993	Memory corruption in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-41992	Memory corruption in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-41991	Memory corruption in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-41064	Arbitrary code execution in Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-41061	Arbitrary code execution in Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-3079	Memory corruption in Skia graphics library of Google Chrome	Google Chrome	9.8
CVE-2023-32409	Heap buffer overflow in WebKit on Apple iOS (Safari)	Apple iOS (Safari)	9.8
CVE-2023-2136	Use-after-free vulnerability in DOM handling of Google Chrome	Google Chrome	9.8
CVE-2023-2033	Integer overflow in rendering engine of Google Chrome	Google Chrome	9.8
CVE-2023-29539	Out-of-bound memory access in WebGL on macOS	Mozilla Firefox	7.5
CVE-2023-29546	Buffer overflow in JavaScript JIT compiler	Mozilla Firefox	7.5

• LibreWolf

LibreWolf Installation on Debian

• Firefox

Firefox ESR Vulnerabilities

Mozilla Vulnerabilities

$ sudo apt install firefox-esr

Copy

• Chromium

Chromium Vulnerabilities

$ sudo apt install chromium

Copy

Torproject - Tor Browser

Tor Browser Vulnerabilities

Everyday TOR

torbrowser-launcher [contrib]

Debian TorBrowser Wiki

Install Tor Browser Outside of Whonix

$ sudo apt install torbrowser-launcher

Copy

$ torbrowser-launcher

Copy

$ torbrowser-launcher --settings

Copy

AppImage

Extensions

Firefox	Chrome
Ublock Origin XBrowserSync Privacy Badger ClearURLs Temporary Containers Canvas Blocker Decentraleyes Terms of Service; Didn't Read LanguageTool jDownloader	Simple Speed Dial Ublock Origin XBrowserSync Privacy Badger Strips Google Analytics LanguageTool Reader View jDownloader

Others - Make

Simple Speed Dial
Bypass Paywalls
Adblock Plus (Firefox, Chrome, Opera, Android): Customizable ad-blocking plugin
Beef Taco (Targeted Advertising Cookie Opt-Out) (Firefox): Sets permanent targeted-advertising opt-out cookies
BetterPrivacy (Firefox): Removes and deletes long-term “super-cookies”
Blur (Firefox): Tracker blocking, password management, and e-mail masking
Decentraleyes (Firefox): Limits tracking by third-party content delivery networks by storing copies of the content locally
Disconnect (Firefox, Chrome, Safari, Opera): Stops 3rd party tracking sites around the web.
Facebook Disconnect (Firefox, Chrome, Opera): Blocks third-party website requests to Facebook to limit Facebook's tracking of users' web activity
Flashblock (Firefox): Allows selective blocking and execution of Flash content.
Flash Control (Chrome): Allows selective blocking and execution of Flash content.
Ghostery (Firefox, Chrome, Safari, Opera): Script and tracker blocking.
HTTPS Everywhere (Firefox, Chrome): Forces HTTPS versions of websites where they are available
Lightbeam (Firefox): Visualization plugin that shows web sites visited, including the third-party web sites that users may not be aware of
NoScript (Firefox, derivatives of Mozilla): Highly customizable plugin to selectively allow Javascript, Java, and Flash to run.
Privacy Badger (Firefox, Chrome, Opera): Blocks third-party scripts and images
Protect My Choices (Chrome): Sets cookies to opt out of targeted advertising
ScriptSafe (Chrome): Script blocking
Self-Destructing Cookies (Firefox): Automatically removes a site's cookies on closing its browser tab
uBlock Origin (Firefox, Chrome, Safari, Opera): Lightweight blocking plug-in for multiple browsers

Extensions Shortcuts

  Examples:
  U-BLOCK ORIGIN (PICK) ALT + SHIFT + P
  READER VIEW (TOGGLE) ALT + SHIFT + T
  ZOTERO CONNECTOR (SAVE) ALT + SHIFT + S

Tips

Reopen with Maximized Browser

To open the browser with a maximized window, use "--start-maximized" as a parameter.


  $ nano /home/user/Desktop/browser.desktop


  [Desktop Entry]
  Exec=/usr/bin/browser --start-maximized %U

Translate from Search Bar

It's possible to translate directly from the browser bar by setting the shortcut to DeepL.


  EN-ES:
  https://deepl.com/en/translator#en/es/%s
  ES-EN:
  https://deepl.com/en/translator#es/en/%s


  EN-DE:
  https://deepl.com/en/translator#en/de/%s
  DE-EN:
  https://deepl.com/en/translator#de/en/%s


  DE-ES:
  https://deepl.com/en/translator#de/es/%s
  ES-DE:
  https://deepl.com/en/translator#es/de/%s

URL Shorteners

Kutt.it

Shlink

02 Cloud Services

Info

https://forum.rclone.org https://reddit.com/r/cloudstorage https://reddit.com/r/DataHoarder https://reddit.com/r/Piracy https://reddit.com/r/Scams

Cloud Privacy

ownCloud : Self-hosted file and sync server
Tahoe-LAFS : Encrypted cloud storage system
Cryptomator (GUI) :
Duplicati (GUI) :

Cloud Providers

Provider	Links	Free Storage	Free Data Download	Encryption	Anonymous	Decentralized	No Signup	Platform Support
Google Drive	GitHub - gdrive	15 GB	No	Yes	No	No	No	Windows, macOS, Linux (via third-party)
MEGA	MEGA MEGA Desktop GitHub - rclone	20 GB	Yes	Yes, End-to-end	Yes	No	No	Windows, macOS, Linux
Oyster Data	Oyster Data	5 GB	No	Yes	No	No	No	Windows, macOS, Linux
Sia	Sia GitHub - SiaFoundation	Depends on storage providers	Yes	Yes, Decentralized	Yes	Yes	Yes	Windows, macOS, Linux
Storj	Storj GitHub - Storj	Depends on storage providers	Yes	Yes, End-to-end	Yes	Yes	Yes	Windows, macOS, Linux
Filecoin	Filecoin GitHub - Filecoin Project	Depends on network providers	Depends on network providers	Yes, End-to-end	Yes	Yes	Yes	Windows, macOS, Linux
Safe Network (MaidSafe)	Safe Network GitHub - MaidSafe	Depends on network providers	Yes	Yes, End-to-end	Yes	Yes	Yes	Windows, macOS, Linux
Arweave	Arweave GitHub - Arweave Team	Depends on network providers	Yes	Yes, End-to-end	Yes	Yes	Yes	Windows, macOS, Linux
Yandex	Yandex rclone Yandex (Backend supported) *Russian	10 GB	No	Yes	No	No	No	Windows, macOS, Linux
IDrive	IDrive IDrive Linux IDrive Scripts rclone IDrive E2	5 GB	No	Yes	No	No	No	Windows, macOS, Linux
TeraBox	TeraBox TeraBox for PC Reddit - TeraBox	1 TB	Yes	Yes	No	No	No	Windows, macOS, Linux
pCloud	pCloud pCloud Linux GitHub - pCloud Console Client	10 GB	Yes	Yes, Optional end-to-end	No	No	No	Windows, macOS, Linux
SugarSync	SugarSync rclone SugarSync (*Not backend supported)	None	No	Yes	No	No	No	Windows, macOS, Linux
Box	Box GitHub - BoxCLI GitHub - rclone	10 GB	No	Yes	No	No	No	Windows, macOS, Linux

03 File Host

File Host

• Unsee

https://unsee.cc

• Rapidgator

https://rapidgator.net

• Nitroflare

https://nitroflare.net

• Uploadgig

https://uploadgig.com

• Mediafire

https://mediafire.com/upgrade/

• UploadFiles

https://ufile.io

• 1Fichier

https://1fichier.com/hlp.html

• Turbobit

https://turbobit.net

• Filescase

https://filescase.com/

• Hex Upload

https://hexupload.org

Anonymous File Sharing Services With Temporary Online Storage

• Tempsend

https://tempsend.com

• WeTransfer

https://wetransfer.com

• Send Anywhere

https://send-anywhere.com

• SendGB

https://sendgb.com

• Volafile

https://volafile.org

• SendSpace

https://sendspace.com

• MyAirBridge

https://myairbridge.com/en/eng

• Gofile

https://gofile.io/welcome

• OTF One Time File

• Bitwarden Send

https://bitwarden.com/products/send

04 Online Utilities

Utilities

Message Header Analysis

Phishing and Email Reputation

Whois and DNS Lookup

Malware and Vulnerability Databases

Archiving Tools

IP and MAC Address Tools

URL and Virus Scanning

File Analysis and Reverse Engineering

Utilities and Development Tools

OSINT and Breach Checking

Translation and Others Tools

6.04 File Management

01 Files and Folders

Synchronize files and folders

$ sudo apt install grsync

Duplicated files

(*by name, size, hash)

dupeGuru

$ sudo apt install dupeguru

fdupes (slow)

$ sudo apt install fdupes

  • Command
  $ fdupes -r dir
  $ fdupes -r -S .

Duplicate Files Finder with md5sum

https://github.com/pixelb/fslint

The package md4sum is a builtin from coreutils

For small files based on hash:


  find . -type f -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

For large files based on size:


  find . -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

*Note: Replace "{}" with the desired file size in bytes.

For large files based on size first and then hash:


  find . -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find . -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

Visualize folder tree

$ sudo apt install tree

  • List directories only
  $ tree -d
  • Control the depth of the tree
  $ tree -d -L 2 .

02 Renamers

Renamers

Online regex tools

• Dencode - https://dencode.com
• Commonly Used Software Development Tools - https://ctool.dev
• Text Fixer - https://textfixer.com
• SS64 Syntax Utils - https://ss64.com
• Tools4noobs - https://tools4noobs.com
• Regex101 - https://regex101.com

File Naming Best Practices

You might consider including some of the following information in your file names, but you can include any information that will allow you to distinguish your files from one another.

Project or experiment name or acronym
Location/spatial coordinates
Researcher name/initials
Date or date range of experiment
Type of data
Conditions
Version number of file
Three-letter file extension for application-specific files

Another good idea is to include in the directory a readme.txt file that explains your naming format along with any abbreviations or codes you have used.

Machine readable

Regular expression and globbing friendly
- Avoid spaces, punctuation, accented characters, case sensitivity
- Easy to compute on
Deliberate use of delimiters

Consider these additional tips as you develop a file naming scheme:

A good format for date designations is YYYYMMDD or YYMMDD. This format makes sure all of your files stay in chronological order, even over the span of many years.
Try not to make file names too long, since long file names do not work well with all types of software.
Special characters such as ~ @ # $ % ^ & * ( ) ` ; < > ? , [ ] { } ' " and | should be avoided.
Illegal characters in Windows file names.
When using a sequential numbering system, using leading zeros for clarity and to make sure files sort in sequential order. For example, use "001, 002, ...010, 011 ... 100, 101, etc." instead of "1, 2, ...10, 11 ... 100, 101, etc."
Do not use spaces. Some software will not recognize file names with spaces, and file names with spaces must be enclosed in quotes when using the command line. Other options include:
- Underscores, e.g. file_name.xxx
- Dashes, e.g. file-name.xxx
- No separation, e.g. filename.xxx
- Camel case, where the first letter of each section of text is capitalized, e.g. FileName.xxx
Periods can be used in file names but consider these points before doing so and proceed cautiously:
- Periods are used in regular expressions.
- Periods at the start of a file name are used to indicate configuration and/or hidden files in a file directory.
- Periods are used to separate file names from file extensions.

Special Symbols

\n is a symbol for new line \t is a symbol for tab \r is for 'return'

Note: \n or \t or \r are interpreted inside of <pre> text </pre>

Special HTML codes

Char	Encoding	Numeric code	Named code	Description
	` `			non-breaking space
	` `			horizontal tab
	` `			line feed
	` `			carriage return / enter
"	`"`	"	"	double quote
'	`'`	'	'	single quote (apostrophe)
<	`<`	<	<	less than
>	`>`	>	>	greater than
&	`&`	&	&	ampersand
©	`©`	©	©	copyright symbol
®	`®`	®	®	registered trademark symbol
™	`™`	™	™	trademark symbol
€	`€`	€	€	euro symbol
À	`À`	À	À	Latin capital letter A with grave accent
é	`é`	é	é	Latin small letter e with acute accent
Ö	`Ö`	Ö	Ö	Latin capital letter O with diaeresis

HTML Encoding and Decoding via Command Line in Linux

HTML Encoding

This will encode the HTML tags and special characters for safe display in HTML if you need to encode special characters for HTML:

echo "&lt;p&gt;This is a paragraph.&lt;/p&gt;" | sed -e 's/&/\&/g' -e 's/</\</g' -e 's/>/\>/g' -e 's/"/\"/g' -e "s/'/\'/g"

HTML Decoding

This will decode HTML entities back to their original characters if you need to decode HTML entities back to their original characters:

echo "&amp;lt;p&amp;gt;This is a paragraph.&amp;lt;/p&amp;gt;" | sed -e 's/&/\&/g' -e 's/&lt;/</g' -e 's/&gt;/>/g' -e 's/&quot;/"/g' -e "s/&apos;/\'/g"

KRename

$ sudo apt install krename

GPRename

$ sudo apt install gprename

Case Styles

Camel case: camelCase

Pascal case: PascalCase

Kebab case: kebab-case

Snake case: snake_case

Dot case: dot.case

Title case: Title Case

Sentence case: Sentence case

Bash's built-in (internal) commands to rename (Debian/GNU Linux)

Safety check before irreversible batch processing rename


  $ for f in *; do echo mv "$f" "$(echo "$f" | tr 'A-Z' 'a-z')"; done


  $ for f in *; do echo mv "$f" "$(echo "$f" | sed -e 's/\([A-Z]\)/-\L\1/g' | sed -e 's/^-//')"; done

Note about uppercase and lowercase with one-liner command in Linux

Its impossible to rename a file in the same directory from uppercase to lowercase with one-liner command due to the way Unix-like systems handle file names. File systems on Unix-like systems (such as ext4, NTFS, etc.) are typically case-sensitive. This means that FILENAME.txt and filename.txt are treated as distinct files. However, you can achieve this by using a temporary directory or by copying the file to a new name and then deleting the original.

Working with files instead of folders in one-liner command.

For example, if you have a file named CamelCase_Example.txt, running the command will rename it to camel-case-example.txt.

Make sure to replace $1 with the actual filename if you're not passing it as an argument to a script or command, and ensure you run the command in the directory where the file is located.


  $ mv
   "$1" "$(echo "$1" | sed 's/\([a-z0-9]\)\([A-Z]\)/\1-\2/g' | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g')"


  $ mv "/dir/CamelCase_Example.txt" "$(echo "/dir/CamelCase_Example.txt" | sed 's/\([a-z0-9]\)\([A-Z]\)/\1-\2/g' | tr '[:upper:]' '[:lower:]' | sed 's/_/-/g')"

One-liner built-in commands to rename

Usage: $ command

• One-liner commands - CamelCase to kebab-case


  $ mv "CamelCaseExample.txt" "$(echo "CamelCaseExample.txt" | sed -e 's/\([A-Z]\)/-\L\1/g' | sed -e 's/^-//')" 
  $ echo "MyDirectoryFileLine" | sed -e 's/\([A-Z]\)/-\L\1/g' 
  $ echo "MyDirectoryFileLine" | sed -e 's/\([A-Z]\)/-\L\1/g' -e 's/^-//' 
  $ echo "MyDirectoryFileLine" | sed -e 's/\([A-Z]\)/-\1/g' -e 's/^-//' 
  $ echo "MyDirectoryFileLine" | sed -e 's/[A-Z]/-\L&/g' -e 's/^-//' 
  $ echo "MyDirectoryFileLine" | sed -e 's/[A-Z]/-\l&/g;s/.//' 
  $ echo "SomeACRONYMInCamelCaseString" | sed -e 's/\([a-z]\)\([A-Z]\)/\1-\L\2/' | sed -e 's/\(.*\)/\L\1/')

Batch processing built-in commands to rename

Usage: "$ cd /folder" and "$ command"


  • Batch Processing - Rename by replacing
  $ for f in *.jpg; do mv "$f" "$(echo "$f" | sed s/IMG/VACATION/)"; done 
  • Batch Processing - Spaces to Dashes 
  $ for f in *; do mv "$f" "$(echo "$f" | tr ' ' '-')"; done 
  • Batch Processing - Uppercase to Lowercase 
  $ for f in *; do mv "$f" "$(echo "$f" | tr 'A-Z' 'a-z')"; done 
  • Batch Processing - Lowercase to Uppercase 
  $ for f in *; do mv "$f" "$(echo "$f" | tr 'a-z' 'A-Z')"; done 
  • Batch Processing - CamelCase to snake_case 
  $ for f in *; do mv "$f" "$(echo "$f" | sed -E 's/([A-Z])/_\1/g' | tr 'A-Z' 'a-z')"; done 
  • Batch Processing - snake_case to CamelCase 
  $ for f in *; do mv "$f" "$(echo "$f" | sed -r 's/(^|_)([a-z])/\U\2/g')"; done 
  • Batch Processing - Title Case (First letter of each word capitalized) 
  $ for f in *; do mv "$f" "$(echo "$f" | sed -E 's/(^|_)([a-z])/\U\2/g' | sed -E 's/_/ /g')"; done 
  • Batch Processing - Title Case with Spaces to snake_case 
  $ for f in *; do mv "$f" "$(echo "$f" | tr ' ' '_' | tr 'A-Z' 'a-z')"; done

Others

Convert Camel case to kebab-case


  • Simple command 
  $ echo "MyDirectoryFileLine" | sed -e 's/\([A-Z]\)/-\L\1/g' -e 's/^-//'

• Batch command


  $ find . -maxdepth 1 -type f -name '*[A-Z]*' -exec bash -c 'mv "$0" "$(echo "$0" | sed -e "s/\([A-Z]\)/-\L\1/g" -e "s/^-//")"' {} \;

Kebab-case to CamelCase

• One-liner command


  $ echo "my-directory-file-line" | sed -r 's/-(.)/\U\1/g'

• Batch command


  $ find . -maxdepth 1 -type f -name '*-*' -exec bash -c 'mv "$0" "$(echo "$0" | sed -r "s/-(.)/\U\1/g")"' {} \;

Kebab-case to snake_case

• One-liner command


  $ echo "my-directory-file-line" | sed 's/-/_/g'

• Batch processing command


  $ find . -maxdepth 1 -type f -name '*-*' -exec bash -c 'mv "$0" "$(echo "$0" | sed "s/-/_/g")"' {} \;

PascalCase to snake_case

• One-liner command


  $ echo "MyDirectoryFileLine" | sed -r 's/([A-Z])/_\L\1/g' | sed 's/^_//'

• Batch processing command


  $ find . -maxdepth 1 -type f -name '*[A-Z]*' -exec bash -c 'mv "$0" "$(echo "$0" | sed -r "s/([A-Z])/_\L\1/g" | sed "s/^_//")"' {} \;

Renamers in shell functions

To incorporate the script into your .bashrc or .bash_profile configuration file, follow these steps:

Open your .bashrc or .bash_profile file using a text editor. For example, you can use nano:

$ ano ~/.bashrc

Copy

Add the script function camel_to_kebab() along with the necessary helper function is_encrypted() to the file. You can copy the entire camel_to_kebab() function along.

Save and exit the text editor. In Nano, you can do this by pressing Ctrl + O to write the changes and Ctrl + X to exit.

Source your updated configuration file to apply the changes immediately:

$ source ~/.bashrc

Copy

or

<pre><code><span>$ </span>source ~/.bash_profile</code></pre>

Copy

You can use it in any directory to convert CamelCase filenames to Title Case. For example:

<pre><code><span>$ </span>camel_to_kebab /file/dir</code></pre>

Copy

Camel to kebab

<pre><code>

camel_to_kebab() { for f in *; do new_name="$(echo "$f" | sed -e 's/([A-Z])/-\L\1/g' | sed -e 's/^-//')" if [ "$f" != "$new_name" ]; then mv "$f" "$new_name" fi done }

Kebab to title case


  find . -maxdepth 1 -type f -name '*-*' -exec bash -c ' kebab_to_title() { echo "$1" | sed -E "s/-/_/g" | awk '\''{ for (i = 1; i <= NF; i++) { if (tolower($i) ~ /^(in|on|at|of|and|or|but|to|the|a|an)$/) { $i = tolower($i) } else { $i = toupper(substr($i, 1, 1)) tolower(substr($i, 2)) } } print $0 }'\'' FS="_" OFS="_" } for file in "$@"; do base="${file%.*}" ext="${file##*.}" ew_name=$(kebab_to_title "$base") if [ "$base" != "$new_name" ]; then mv -v "$file" "${new_name}.${ext}" fi done ' _ {} +

Camel to title case


  camel_to_title() { convert_to_title() { echo "$1" | sed -E 's/([a-z])([A-Z])/\1 2/g' | awk '{ for (i = 1; i <= NF; i++) { if (tolower($i) ~ /^(in|on|at|of|and|or|but|to|the|a|an)$/ && i != 1) { $i = tolower($i) } else { $i = toupper(substr($i, 1, 1)) tolower(substr($i, 2)) } } print $0 }' OFS=" " } for file in *; do if [[ -f "$file" ]]; then base="${file%.*}" ext="${file##*.}" new_name=$(convert_to_title "$base") if [ "$base" != "$new_name" ]; then mv -v "$file" "$new_name.${ext}" fi fi done }

Rename

$ sudo apt install rename


  • Commands for rename 
  • Syntax 
  $ rename [options] 's/[pattern]/[replacement]/' [file name] 
  • Replacing the blank space with an underscore (_) 
  $ rename -v 'y/ /\_/' *.pdf 
  $ rename -v 'y/ /\_/' ~/Downloads/* 
  $ rename -v 'y/ /\_/' ~/Downloads/*.pdf 
  $ rename -v 'y/\n/\_/' ~/Downloads/*.pdf 
  $ rename -v 'y/\-/\_/' ~/Downloads/*.pdf 
  • Commands to rename to numbered order 
  $ cd /Files 
  • Test the output before (* -n) 
  $ rename -n 's/.+/our $i; sprintf("input%03d.png", 1+$i++)/e' * 
  • Apply the change 
  $ rename 's/.+/our $i; sprintf("input%03d.png", 1+$i++)/e' * 
  • Delete a Part of a Filename 
  $ rename -v 's/example//' *.pdf 
  • Convert Uppercase to Lowercase Characters #FAIL 
  $ rename -v 'y/[A-Z]/[a-z]/' *.PDF 
  $ find my_dir -type f -execdir rename 'y/A-Z/a-z/' {} \; 
  • Convert Lowercase to Uppercase Characters #FAIL 
  $ rename -v 'y/[a-z]/[A-Z]/' *.pdf 
  • Convert to Camel case 
  $ rename 's/ /_/g' *

Metadata Renamer

• Rename files into directories according to metadata contained in.
• Exiftool Pseudo Tags - https://exiftool.org/filename.html
• Illegal characters in Windows file names.

• Basic of metadata

$ exiftool -a input.pdf

$ exiftool -a -s input.pdf

$ exiftool -a -G1 input.pdf

$ exiftool -G1 -s 'input.pdf' | grep '\[PDF\]'

$ pdfinfo input.pdf

$ pdfinfo -meta input.pdf

-a shows all metadata tags across all groups (EXIF, IPTC, XMP, etc.).
-s shows metadata tags in a short format, without grouping by metadata type.
-a -G1 shows all tags, including those not normally extracted, and groups them by family 1 (e.g., PDF tags grouped under [PDF]).

• Rename by creation date and time tags [20060327_1058-2.jpg]

$ exiftool -d %Y%m%d_%H%M%%-c.%%e "-filename<CreateDate" /file.jpg

• Rename by title tag [title.pdf]

$ exiftool '-filename<Title ' /input.pdf

• Rename by title and author tags [Title - Author .pdf]

$ exiftool '-filename<$Title ${Author}.%e ' /input.pdf

• Rename by title, author and date tags [Title - Author (Year).pdf]

$ exiftool '-filename<$Title - ${Author} (${Date#;DateFmt( "%Y")}).%e ' /input.pdf

$ exiftool '-filename<$Title - ${Author} (${CreationDate#;DateFmt( "%Y")}).%e ' /input.pdf

• Recursively

$ exiftool -r '-filename<${Title} - ${Author}.%e ' /DIR -ext pdf

$ exiftool -r '-filename<${Title} - ${Author} (${CreationDate#;DateFmt( "%Y")}).%e ' /DIR -ext pdf

• Rename by title, author and date tags [Title - Author (Year).pdf] and insert snake case format

$ exiftool -r '-filename<${Title;s/ /_/g}_-_${Author;s/ /_/g}_(${CreationDate#;DateFmt( "%Y")}).%e ' /DIR -ext pdf

03 Backup Tools

https://github.com/bit-team/backintime https://github.com/teejee2008/timeshift https://github.com/restic/restic

Recover plan

Backup

Folders and Files Backup

GRSYNC (GUI)

$ sudo apt install grsync

Copy

GRSYNC is a graphical interface for the rsync command line program. It can be used for local directory synchronization.

Copy files and directories in CLI

Full backup

Basic command

$ cp ~/.config/example/ /media/backup

Copy

Incremental backup

Basic command

$ cp -vur ~/.config/example/ /media/backup

Copy

*This will preserve the files permissions/ownership.

∙ To get the progress information

$$ cp -a ~/.config/example/ /media/backup

Transfer files in CLI

CURL

$ sudo apt install curl

Copy

curl is mainly used for transferring data over various protocols.
curl -o destination FILE://source downloads a file from a URL (FILE://source) to a destination but doesn't preserve permissions and ownership.

  $ curl -o destination FILE://source

*This won't preserve the files permissions and ownership.

WGET

$ sudo apt install wget

Copy

wget is a free utility for non-interactive download of files from the web.
wget URL downloads the file from the specified URL and saves it in the current directory.
wget -P /path/to/directory URL downloads the file from the specified URL and saves it in the specified directory.
wget -r URL recursively downloads all files from the specified URL.

  $ wget URL
  $ wget -P /path/to/directory URL
  $ wget -r URL

*WGET supports many options and can handle various download scenarios such as recursive downloading, downloading from FTP, resuming downloads, and mirroring websites.

RSYNC

https://wiki.archlinux.org/title/Rsync

$ sudo apt install rsync

Copy

rsync is a powerful tool for copying and syncing files locally or remotely.
-ah options preserve permissions and ownership (-a for archive mode, -h for human-readable output).
--progress shows the transfer progress and speed.
--info=progress2 provides detailed progress information.
--remove-sent-files is useful for moving files after successful transfer.


  • Copying 
  $ rsync -ah --progress ~/source /backup/destination 
  $ rsync -ah --info=progress2 ~/source /backup/destination 
  $ rsync -rah --info=progress2 ~/source /backup/destination 
  • Moving 
  $ rsync -ravzP --remove-sent-files ~/source /backup/destination 
  • Copy a File or Directory from Local to Remote Machine 
  $ rsync -ravP Directory/ server@192.168.1.56:/home/server

SSH

$ sudo apt install openssh-client

Copy

scp (secure copy) is used for securely transferring files between hosts on a network.
scp source_file user@remote_host:/path/to/destination transfers a file from the local machine to a remote host.
scp user@remote_host:/path/to/source_file destination transfers a file from a remote host to the local machine.
scp -r source_directory user@remote_host:/path/to/destination transfers a directory and its contents recursively from the local machine to a remote host.
scp -r user@remote_host:/path/to/source_directory destination transfers a directory and its contents recursively from a remote host to the local machine.

  $ scp source_file user@remote_host:/path/to/destination
  $ scp user@remote_host:/path/to/source_file destination
  $ scp -r source_directory user@remote_host:/path/to/destination
  $ scp -r user@remote_host:/path/to/source_directory destination

*SSH also supports many other options and utilities like rsync for more advanced file transfer operations.

DD

dd is a command-line utility for copying and converting files and block-level data.
It preserves file permissions and ownership by default.
status=progress option shows the progress of data transfer.

  $ dd if=~/source of=/backup/destination status=progress

PV

$ sudo apt install pv

Copy

pv is a command-line tool to monitor the progress of data through a pipeline
pv ~/source > /backup/destination copies the content of ~/source to /backup/destination but doesn't preserve file permissions and ownership.

  $ pv ~/source > /backup/destination

*This won't preserve the files permissions and ownership.

Disk Imaging and Cloning

Clonezilla

The Free and Open Source Software for Disk Imaging and Cloning

https://clonezilla.org//clonezilla-live-doc.php

Changing disk name

$ cnvt-ocs-dev -d /home/partimag 'image' 'sda3' 'sda2'

Data Integrity

https://debian.org/doc/manuals/securing-debian-manual/ch04s17.en.html#check-integ

04 Extracting Files

https://wiki.debian.org/Compression https://cvedetails.com/vendor/9220/7-zip.html https://cvedetails.com/vendor/1914/Rarlab.html https://cvedetails.com/vendor/72/GNU.html

All-in-one Extract Incorporated in Shell (Shell Function)

To set up the extract function, start by installing the required tools.

Simplified installation list:

$ sudo apt install -y unzip tar unrar

Copy

Advanced installation list:

$ sudo apt install -y tar gzip 7zip unrar zlib1g bzip2 xz-utils tarlz

Copy

Incorporating the Script into Your Shell Configuration

Simplified Extract Function into Your Shell Configuration

Open your .bashrc or .bash_profile file using a text editor. For example, you can use nano:
```
$ sudo apt install -y unzip tar unrar
```
Copy
```
$ nano ~/.bashrc
```
Copy

Add the extract() function to the file:


  # Extract common archive files by file extension
  extract() {
    if [ -f "$1" ]; then
        case "$1" in
            *.tar.gz|*.tgz) tar xzf "$1" ;;
            *.tar) tar xf "$1" ;;
            *.tar.bz2|*.tbz2) tar xjf "$1" ;;
            *.zip) unzip "$1" ;;
            *.rar) unrar x "$1" ;;
            *)
                echo "Unsupported file format: '$1'"
                ;;
        esac
    else
        echo "'$1' is not a valid archive file."
    fi
  }

Save and exit the text editor. In Nano, you can do this by pressing Ctrl + O to write the changes and Ctrl + X to exit.
Source your updated configuration file to apply the changes immediately:
```
$ source ~/.bashrc
```
Copy
or
```
$ source ~/.bash_profile
```
Copy

Usage

You can now use the extract function directly from your terminal to extract archive files. For example:

$ extract example.tar.gz

Copy

Advanced Extract Function into Your Shell Configuration

Open your .bashrc or .bash_profile file using a text editor. For example, you can use nano:
```
$ sudo apt install -y tar gzip 7zip unrar zlib1g bzip2 xz-utils tarlz
```
Copy
```
$ nano ~/.bashrc
```
Copy

Add the extract() function along with the is_encrypted() helper function to the file. You can copy the following entire script:


  # Function to check if a file is encrypted
  is_encrypted() {
    file "$1" | grep -q "encrypted"
  }
Function to securely prompt for password

prompt_password() {
local password=""
local prompt="Enter password for $1: "
# Prompt for password without echoing to the terminal
while IFS= read -rs -p "$prompt" char; do
    # Exit loop when Enter key is pressed
    if [[ -z $char ]]; then break; fi
    password+="$char"
    prompt='*'
done
echo "$password"

}
Extract common archive files by file extension

extract() {
if [ -f "$1" ] ; then
case "$1" in
.tar.gz|.tgz) tar xzf "$1" ;;
.tar|.tar.xz) tar xf "$1" ;;
.tar.bz2|.tbz2) tar xjf "$1" ;;
*.xz) unxz "$1" ;;
*.zip)
if is_encrypted "$1"; then
password=$(prompt_password "$1")
unzip -P "$password" "$1"
else
unzip "$1"
fi
;;
*.Z) uncompress "$1" ;;
*.tar.zst) tar -I=unzstd -xf "$1" ;;
*.zst) unzstd "$1" ;;
*.7z)
if is_encrypted "$1"; then
password=$(prompt_password "$1")
7z x -p"$password" "$1"
else
7z x "$1"
fi
;;
.tar.gz.gpg|.tgz.gpg)
gpg -d "$1" | tar xzf -
;;
*.tar.gpg)
gpg -d "$1" | tar xf -
;;
.tar.bz2.gpg|.tbz2.gpg)
gpg -d "$1" | tar xjf -
;;
*.zip.gpg)
password=$(prompt_password "$1")
gpg -d "$1" | unzip -P "$password" -
;;
*.xz.gpg)
gpg -d "$1" | unxz -
;;
*)
echo "Unsupported file format: '$1'"
;;
esac
else
echo "'$1' is not a valid archive file."
fi
}

Save and exit the text editor. In Nano, you can do this by pressing Ctrl + O to write the changes and Ctrl + X to exit.
Source your updated configuration file to apply the changes immediately:
```
$ source ~/.bashrc
```
Copy
or
```
$ source ~/.bash_profile
```
Copy

Usage

You can now use the extract function directly from your terminal to extract archive files. For example:

$ extract example.tar.gz

Copy

• TAR (.gz , .tar and .tar.gz)

  Commands for .tar archives
  • How to create an .tar file with gzip archiver:
  $ tar –cvf outarchive.tar ~/Documents
  • How to decompress a .tar file with with gzip:
  $ tar -xvf archive.tar

  Commands for .tar.gz archives
  • How to create an .tar.gz file
  $ tar –cvzf outarchive.tar.gz ~/Documents
  • To list the contents of a .tar.gz file:
  $ tar –tzf archive.tar.gz
  • How to decompress a .tar.gz file
  $ tar –xvzf archive.tar.gz
  $ tar –xvzf archive.tar.gz –C /home/user/Downloads

• GZIP (.gz , .tar and .tar.gz)

$ sudo apt install gzip

Copy

  Commands for .gz archives
  • How to create an .gz file
  $ gzip outarchive.gz indoc1.pdf
  • How to decompress a .gz file
  $ gunzip archive.gz

• 7Z (.7z and .zip)

https://7-zip.org
https://cvedetails.com/vendor/9220/7-zip.html

$ sudo apt install p7zip-full

Copy

  Commands for .7z archives
  • How to create an .7z file
  $ 7z a outarchive.7z indoc1.pdf
  • How to decompress a .7z file
  $ 7z x archive.7z

  Commands for .zip archives
  • How to create an zip file
  $ 7z a outarchive.zip indoc1.pdf
  • How to decompress a zip file
  $ 7z x archive.zip

  Commands for encrypted .7z and .zip archives
  • How to create an encrypted .zip file
  $ 7z a -p -t7z -scrc=AES256 archive.7z /input/directory
  $ 7z a -p -tzip -scrc=AES256 outarchive.zip indoc1.pdf inpdoc2.pdf
  $ 7z a -p -tzip -scrc=AES256 archive.zip /input/directory
  • How to create an encrypted header .7z file (only)
  $ 7z a -p -mhe=on -scrc=AES256 archive.7z input_dir
  $ 7z a -p -mhe=on -scrc=AES256 /output/archive.7z /input/directory
  • How to decompress a .7z and .zip file that is encrypted
  $ 7z x archive.zip

*Encrypted header: no file list contents visible without the password

• RAR (.rar)

$ sudo apt install unrar-free

Copy

  Commands for .rar archives (*proprietary: extract only)
  • How to decompress a rar file
  $ unrar e ~/Downloads/filename.rar ~/Downloads/
  • How to decompress a rar file encrypted
  $ unrar-free -x ~/Downloads/filename.rar ~/Downloads/
  • How to decompress a rar file encrypted parts, only unrar the first part01.rar and it goes itself for the rest. Navigate to the directory containing the file:
  $ cd /path/to/directory/
  $ unrar-free -xp /part01.rar ~/Downloads/

• ZIP (.zip)

https://infozip.sourceforge.net

$ sudo apt install zip unzip

Copy

  Commands for .zip archives
  • Add file.txt to z.zip (create z if needed)
  $ zip z file.txt
  • Zip all files in current dir:
  $ zip z *
  • Zip files in current dir and subdirs also:
  $ zip -r z .
  • How to decompress a .zip file:
  $ unzip ~/Downloads/filename.zip
  • How to unzip multiple .zip files:
  $ unzip '*.zip'
  • How to decompress a .zip file to directory:
  $ unzip filename.zip -d /path/to/directory
  $ unzip -d file file.zip
  • Unzip Multiple Files (using single quote or backslash)
  $ unzip '*.zip'
  $ unzip \*.zip
  • Locale encoding name error
  $ unzip -I (encoding) (FILE_PATH) -d (Destination)
  $ unzip -I UTF-8 Desktop.zip

  Commands for encrypted .zip archives
  • How to create an encrypted .zip file
  $ zip -e filename.zip ~/Downloads/
  • How to decompress a encrypted .zip file
  $ unzip ~/Downloads/filename.zip
  • How to decompress a encrypted .zip file to directory
  $ unzip ~/Downloads/filename.zip -d ~/Downloads/

6.05 Media Management

01 Image Tools

Basic Image Editors

Gthumb

https://gitlab.gnome.org/GNOME/gthumb
https://cvedetails.com/vendor/283/Gnome.html

$ sudo apt install gthumb

Imagemagick

https://imagemagick.org
https://cvedetails.com/vendor/1749/Imagemagick.html

$ sudo apt install imagemagick

(GUI or CLI)

Advanced Image Editors

GIMP

https://gimp.org
https://cvedetails.com/vendor/9605/?q=Gimp

"Whether you are a graphic designer, photographer, illustrator, or scientist, GIMP provides you with sophisticated tools to get your job done."

$ sudo apt install gimp

Inkscape

https://inkscape.org
https://cvedetails.com/vendor/3488/Inkscape.html

"Inkscape is a Free and open source vector graphics editor for GNU/Linux, Windows and macOS. It offers a rich set of features and is widely used for both artistic and technical illustrations such as cartoons, clip art, logos, typography, diagramming and flowcharting."

Other

Memegen (Open Source)

An API to programmatically generate memes based solely on requested URLs

https://github.com/jacebrowning/memegen
https://memegen.link

imgflip (Free)

https://imgflip.com/memegenerator

02 Media Players

MPV

$ sudo apt install mpv

Copy

Shortcuts - https://github.com/mpv-player/mpv/blob/master/DOCS/man/mpv.rst#keyboard-control

Window Geometry - https://mpv.io/manual/master/#options-geometry

Video Autofit - https://mpv.io/manual/master/#options-autofit

MPV Config

Config - https://github.com/mpv-player/mpv/blob/master/etc/mpv.conf

Coping basic MPV configs

$ cp -r /usr/share/doc/mpv/ ~/.config/mpv/

Copy

Editing only MPV configuration file (.conf)

$ cp /usr/share/doc/mpv/mpv.conf.gz ~/.config/mpv/

Copy

$ gzip -d ~/.config/mpv/mpv.conf.gz

Copy

$ ano ~/.config/mpv/mpv.conf

Copy

Examples:

save-position-on-quit=yes no-border volume-max=125 geometry=50%x96%

To automatically save the current playback position on quit, start mpv with --save-position-on-quit, or add save-position-on-quit=yes to the configuration file.

Set volume-max=value in your configuration file to a reasonable amount, such as volume-max=150 , which then allows you to increase your volume up to 150%, which is more than twice as loud. Increasing your volume too high will result in clipping artefacts. Additionally (or alternatively), you can utilize dynamic range compression with af=acompressor .

MPV Read-me and examples

$ sudo gzip -d /usr/share/doc/README.md.gz ~/

Copy

$ ls ~/.config/mpv/examples/lua/

Copy

Editing only MPV keybindings (input.conf)

$ cp /usr/share/doc/mpv/input.conf.gz ~/.config/mpv/

Copy

$ gzip -d ~/.config/mpv/input.conf.gz

Copy

$ ano ~/.config/mpv/input.conf

Copy

*It's recommended to use mplayer-input.conf as reference-only. To use it save as input.conf .

VLC

$ sudo apt install vlc

Copy

GNOME Media Player

$ sudo apt install totem

Copy

03 Video Editors

Video Editors

Handbrake

$ sudo apt install handbrake

$ sudo apt install handbrake-cli

FFMpeg

$ sudo apt install ffmpeg

Avidemux2

Avidemux2 - Simple video editor

Online editors:

Desktop Recording

OBS Studio

$ sudo apt install -y obs-studio

Simple Screen Recorder

$ sudo apt install -y simplescreenrecorder

YT-DLP - A feature-rich command-line audio/video downloader

https://github.com/yt-dlp/yt-dlp

$ sudo apt install yt-dlp

Download YouTube videos


  • Commands
  • Download a video or playlist
  $ yt-dlp [URL]
  $ yt-dlp -F [URL]
  $ yt-dlp -f 247 [URL]
  $ yt-dlp -f "best[height <=480] " [URL]
  $ yt-dlp -f "best[height<=480] " [URL]
  $ yt-dlp -f worstvideo [URL]
  $ yt-dlp -o 'qwerty' [URL]
  • Download with metadata
  $ yt-dlp -o '%(title)s by %(uploader)s on %(upload_date)s in %(playlist)s.%(ext)s' [URL]
  $ yt-dlp --write-description --write-info-json --write-annotations --write-sub --write-thumbnail [URL]
  • Download audio-only
  $ yt-dlp -x --audio-format mp3 [URL]

FFmpeg editor

https://trac.ffmpeg.org/wiki/Encode/H.264

• Compressing videos

$ ffmpeg -i input.ext output.mp4

$ ffmpeg -i input.ext -b:v output.mp4

$ ffmpeg -i input.ext -vf scale=1280:720 output.mp4

$ ffmpeg -i input.ext -c:v libx265 output.mp4

$ ffmpeg -i input.mp4 -vcodec h264 -acodec mp2 output.mp4

$ ffmpeg -y -i input.mp4 -vcodec h264 -acodec mp3 output.mp4

$ ffmpeg -y -i input.mp4 -vcodec h264 -acodec aac output.mp4

$ ffmpeg -i input.mp4 -vcodec h264 -b:a 96k output.mp4

$ ffmpeg -i input.mp4 -vcodec h264 -b:v 1000k -acodec mp3 output.mp4

$ ffmpeg -i input.mp4 -vcodec libx265 -acodec aac -crf 23 output.mp4

$ ffmpeg -i input.mp4 -c:v libx265 -preset ultrafast -crf 28 -c:a aac -b:a 250k output.mp4

$ ffmpeg -i input.mov -c:v libx265 -preset veryfast -tag:v hvc1 -vf format=yuv420p -c:a copy output.mp4

• Compressing video removing sound (to disable audio you must use -an)

$ ffmpeg -i input.mp4 -vcodec h264 -an output.mp4

• Converting videos

$ ffmpeg -y -i input.wmv output.mp4

$ ffmpeg -i input.mp4 -vf "scale=-2:240 " output.mp4

$ ffmpeg -i input.wmv -c:v libx264 -crf 23 output.mp4

$ ffmpeg -i input.wmv -c:v libx264 -crf 23 -c:a aac -q:a 100 output.mp4

$ ffmpeg -i input.wmv -c:v libx264 -crf 23 -c:a aac -strict -2 -q:a 100 output.mp4

$ ffmpeg -i input.wmv -c:v libx264 -crf 23 -profile:v high -r 30 -c:a aac -q:a 100 -ar 48000 output.mp4

• Scaling down the size of the MP4

$ ffmpeg -i input.mp4 -s 1280x720 -acodec copy -y output.mp4

$ ffmpeg -i input.mp4 -vf "scale=-2:720 " -c:v libx264 -crf 20 -preset slow -c:a copy output.mp4

$ ffmpeg -i input.mp4 -s 1920x1080 -c:v libx265 -preset ultrafast -crf 28 -c:a aac -b:a 250k output.mp4

$ ffmpeg -i input.mp4 scale=1080:1920,format=yuv420p -c:v libx265 -preset veryfast -tag:v hvc1 -b:v 800k -bufsize 1200k -vf -b:a 128k output.mp4

$ ffmpeg -i input.mp4 -c:v libx265 -preset veryfast -tag:v hvc1 -vf format=yuv420p -c:a copy output.mp4

$ ffmpeg -i input.mp4 -c:v libx265 -preset veryfast -tag:v hvc1 -b:v new_bitrate -vf scale=new_width:new_height,format=yuv420p -c:a copy output.mp4

In batch compress MP4

In batch compress MP4 with ffmpeg

  • One-line convert script
  $ for file in *.mp4; do ffmpeg -i "$file " -vf "scale=-2:240 " "Output-${file%.*}.mp4 "; done

04 Audio Editors

Audacity

$ sudo apt install audacity

<a href="https://youtube.com/watch?v=fSmmaAD8XD8 "><img src="https://markdown-videos-api.jorgenkh.no/url?url=https%3A%2F%2Fyoutube.com%2Fwatch%3Fv%3DfSmmaAD8XD8 " alt="Audacity Step-by-Step Tutorial for Beginners (2024) " title="Audacity Step-by-Step Tutorial for Beginners (2024) "></a>
<a href="https://youtube.com/watch?v=bsZOHcDDKeQ "><img src="https://markdown-videos-api.jorgenkh.no/url?url=https%3A%2F%2Fyoutube.com%2Fwatch%3Fv%3DbsZOHcDDKeQ " alt="How to make voice sound better with Audacity (2024) " title="How to make voice sound better with Audacity (2024) "></a><br>

6.06 Artificial Intelligence

ChatGPT

Visit our repo tree: 04_SOFTWARE/06_Artificial_Intelligence

Open-Source LLMs Locally

Feature	GPT4All	privateGPT	localGPT
Aboout	Privacy-oriented software for chatting with large language models that run on your own computer.	Interact with your documents using the power of GPT, 100% privately, no data leaks	Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.
Resource Requirements	- CPU: 4 cores recommended - RAM: 16 GB recommended - Disk: Varies by model size (10-50 GB) - GPU: Optional, improves performance	- CPU: 4 cores recommended - RAM: 16 GB recommended - Disk: 10-20 GB - GPU: Optional, improves performance	- CPU: 4 cores recommended - RAM: 16 GB recommended - Disk: Varies by model size (10-50 GB) - GPU: Recommended for large models
Installation Complexity	Medium - Requires downloading models and setting up environment	Medium - Python environment setup and model downloading	Medium - Requires setting up Python environment and dependencies
Supported Models and Characteristics	- GPT-3: 175B parameters, state-of-the-art performance, Apache 2.0 - GPT-J: 6B parameters, open-source alternative to GPT-3, Apache 2.0 - GPT-Neo: 2.7B, 1.3B parameters, lightweight models, MIT	- GPT-3: 175B parameters, high accuracy and versatility, Apache 2.0 - LLaMA: Various sizes, efficient and scalable, custom licenses (research use only)	- GPT-3: 175B parameters, high accuracy and versatility, Apache 2.0 - GPT-Neo: 2.7B, 1.3B parameters, lightweight models, MIT - BERT: Various sizes, strong at understanding context, Apache 2.0
Community Support	- Support: Active - Forums, GitHub issues, community forums - Documentation: Comprehensive - Detailed setup and usage instructions available	- Support: Active - GitHub discussions, community contributions - Documentation: Good - Includes setup guide and usage examples	- Support: Active - GitHub issues and community support - Documentation: Good - Detailed README with setup and usage instructions
License	Apache 2.0 - Allows commercial use - Permissive, widely used - Requires attribution and disclaimer	MIT - Highly permissive - Allows commercial use - No warranty or liability	MIT - Highly permissive - Allows commercial use - No warranty or liability

6.07 Metaverse

Vircadia - Open Source Metaverse Platform

https://vircadia.com

Overte - Open source virtual worlds and social VR

https://overte.org

6.18 Tips and Tricks

Visualize folder tree

$ sudo apt install tree

Copy

  • List directories only 
  $ tree -d
  • Control the depth of the tree 
  $ tree -d -L 2 .

Schedule a Shutdown in Command Line

Shutdown at specific time

  $ sudo shutdown -h 23:59

Shutdown at specific date and time

Since 24 hours (24×60=1440 minutes), you could adapt.

  $ sudo shutdown +1440

Execute shutdown +60 at a specific time and day:

  $ sudo apt install at 
 
  $ sudo echo "shutdown +767" | at 08:46am 2024-09-11

Set an Alarm from the Terminal

  $ sleep 5m && mpv alarm.mp3

Use Terminal as a Stopwatch

  $ time cat CTRL + C

Commands for you to find large files

  $ df -h 
  $ df -h /home/$USER/Downloads 
  $ du -h ~/Downloads | sort -rh | head -n 10 
  $ find ~/Downloads -type f -size +3G -exec du -h {} + | sort -rh 
  $ find /home/$USER/Downloads -xdev -type f -size +1G 
  $ find . -size +1M -printf "%p \t%k kb\n" | sort -k2n
  $ find  -type f  -exec  du -Sh {} +  |  grep .pdf | sort -rh  |  head -n 50

Find the largest packages

$ dpkg-query --show --showformat='${Package;-50}\t${Installed-Size}\n' | sort -k 2 -n | grep -v deinstall | awk '{printf "%.3f MB \t %s\n", $2/(1024), $1}'

Editig text with sed in terminal

Be cautious of using -n (quiet) with -i (in-place): the former disables aut in-place omatic printing of lines and the latter changes the file in-place without a backup. Make a backup before editing -i.bak.

$ sed --options [optional SCRIPT] [INPUT STREAM]

Copy

  • Append line after match 
  $ sed '/option/a Hello World' *.txt
  • Insert line before match 
  $ sed '/option\/i Hello World' *.txt
  • Append text after a line for the specific match only 
  $ sed '0,/option\/a Hello World' *.txt
  • Append text before a line for the specific match only 
  $ sed '0,/option\/i Hello World' *.txt
  • Inserting a text file. Content of text1 after the first line of text2.txt 
  $ sed -i -e '1r text1' text2.txt

  • Append string in a multiline text 
  $ sed -i '/pattern/ r snippet.txt' filename
  • Batch inserting specific text after pattern in .txt files 
  $ sed -i '/SearchPattern/a New Text' *.txt 
  $ find . -name '*.txt' -exec sed -i -e 's/textp1/textp2 text.txt/g' {} \;

  • Replacement syntax 
  $ sed s/regexp/replacement/[flags]
   • Replace closed with open on lines containing code 
  $ sed '/code/ s/closed/open/' text.txt
   • Replace all closed with open on lines containing code 
  $ sed '/code/ s/closed/open/g' text.txt
   • Replace all occurrences of ‘hello’ to ‘world’ in the file input.txt: 
  $ sed 's/hello/world/g' input.txt > output.txt

  • Batch removing specific text in .txt files, the –i option tells to sed make the change in place, inside the file itself 
  $ sed -i 's/text//g' *.txt
   • Batch making a backup before editing the file, by specifying a suffix when using –i 
  $ sed -i.bak 's/word//g' *.txt

• Convert Lowercase to Uppercase Characters

$ sed -i 's/.*/\U&/' text.txt

$ sed 's/[a-z]/\U&/g' < ./myfile.txt

$ echo "HELLO WORLD " | sed 's/[A-Z]/\L&/g'

• Convert Uppercase to Lowercase Characters

$ echo "hello world" | sed 's/[a-z]/\U&/g'

• Remove leading spaces and tabs from line

$ sed 's/^[ \t]*//' text.txt

• Remove all digits in a range

$ sed 's/[0-9]//g' text.txt

  *Note, in terminal: 
  Alt + u Make uppercase from the cursor position to the end of the word. 
  Alt + l Make lowercase from the cursor position to the end of the word/text.

Reopen with maximized browser

To open browser with maximized window use "--start-maximized" as a parameter.

$ nano /home/user/Desktop/browser.desktop [Desktop Entry] Exec=/usr/bin/browser --start-maximized %U

Geting windows sizes in shell

  $ wmctrl -lG

Changing default editor for `visudo`

By default, Linux systems use the $VISUAL or $EDITOR environment variables (usually defined in your ~/.bashrc file or /etc/profile ) as the default editor the visudo command. If you'd prefer to use a different editor, such as ano , you can use either of these methods.

To temporarily use a different editor, run:
```
$ sudo EDITOR=/path/to/editor visudo
```
For example, to use ano , you would run:
```
$ sudo EDITOR=nano visudo
```
To permanently change the default editor, edit the /etc/sudoers file (you can use the temporary method above!) and add the following line to the file near the top, but after Defaults env_reset :
```
Defaults editor=/path/to/editor 
```

Reference: https://unix.stackexchange.com/questions/4408/how-to-set-visudo-to-use-a-different-editor-than-the-default-on-fedora

Test if a Bash Shell Built-in (Internal) Command or External Program Package

References:

Bourne Shell builtins are internal commands inherited from the original Bourne Shell (1977).

Why use built-in commands? They do not depend on granting permissions to install packages, work well on reduced sets like Linux embedded hardware (IoTs), reduce the attack surface, and offer better performance.


  • See all bash builtins
  $ help
  $ help help
  $ help -s echo
  $ help -d echo


  • Other forms to show the list of builtin commands
  $ enable -a
  $ compgen -b


  • Test using type command 
  $ type -a cd
  $ type -t cd
  $ type -a ls
  $ type -t ls
  $ type -t test
  $ type -t echo
  $ type -t apt 
  $ type -t grep 
  $ type -t for 
  • Test using builtin command 
  $ builtin cd 
  $ builtin ls 
  $ builtin test 
  $ builtin echo 
  $ builtin apt 
  $ builtin grep 
  $ builtin for 
  • Test using command 
  $ command -V cd 
  $ command -V ls

Changing terminal command output language

  • For just a single command, from native language to english 
  $ LC_ALL=C man ls 
  $ LC_ALL=C sudo apt update

Grep - Global regular expression search

  • Commands
  $ man find | grep -A 4 -- '-size' 
  • 
  $ man find | grep -e '-size'

6.19 Entertainment

Steam OS

Steam OS

Kodi.tv

Kodi.tv

Plex Media Server

https://plexopedia.com

$ sudo apt install plexmediaserver

LibreELEC.tv

LibreELEC.tv
HOW-TO:Install Kodi on Raspberry Pi

Linux distribution that transforms devices in consoles

Batocera

7. DEV SETUP

👷🛠️UNDER CONSTRUCTION🚧🏗

7.01 System Tweaks

Visit our repo tree: 05_DEVELOPMENT

Sytem Tweaks

Terminal

Window Shortcuts

Passwords

Mananger

SSH

Interface

7.02 IDEs

IDEs

• NeoVim

https://neovim.io
https://neovim.io/doc/user/starting.html
https://neovim.io/doc/user/usr_01.html#vimtutor
https://github.com/neovim/nvim-lspconfig#suggested-configuration
https://youtube.com/watch?v=RZ4p-saaQkc
https://github.com/rockerBOO/awesome-neovim
https://reddit.com/r/neovim

∙ Setups

https://github.com/nvim-lua/kickstart.nvim
https://github.com/LazyVim/LazyVim
https://github.com/LunarVim/LunarVim
https://github.com/NvChad/NvChad
https://spacevim.org/

∙ Plugins

https://siddharta.me/configuring-neovim-as-a-python-ide-2023.html
https://thevaluable .dev/vim-php-ide/

• VSCodium

https://github.com/VSCodium/vscodium
https://reddit.com/r/vscodium

∙ Extensions

• Sublime-text®

https://sublimetext.com/docs/linux_repositories.html
https://reddit.com/r/sublimetext

7.03 Git & GitHub

Git & GitHub

•

7.04 Languages

Languages

• C

  #include <stdio.h>
  
  int main() {
      printf("Hello, World!\n");
      return 0;
  }

• Python

  print("Hello, World!")

• Rust

  fn main() {
      println!("Hello, World!");
  }

• PHP

  <?
  php echo 'Hello, World!'; ?>

• Ruby

  puts "Hello, World!"

7.05 Others

Others

• Deploy

• Database

• Projects Folder

• Team

8. SYSADMIN

👷🛠️UNDER CONSTRUCTION🚧🏗

8.01 Sysadmin

Visit our repo tree: 06_SYSADMIN

9. TROUBLESHOOTING

👷🛠️UNDER CONSTRUCTION🚧🏗

9.01 Linux Community

Linux Community

https://forums.debian.net
https://unix.stackexchange.com
https://forum.linuxfoundation.org
https://askubuntu.com
https://docs.kernel.org
https://linuxquestions.org
https://superuser.com
https://stackoverflow.com
https://howtoforge.com
https://security.stackexchange.com
https://data.stackexchange.com
https://elinux.org
https://hardforum.com
https://snbforums.com
https://reddit.com/r/debian
https://reddit.com/r/linuxquestions
https://reddit.com/r/sysadmin

9.02 Log Managemnt

Secure Log Management and Advanced Logging on Debian

Securely Managing Logs in Debian

Separate Log Storage: Store logs on a separate read-only disk or partition.
File Permissions: Set appropriate permissions using chmod and chown.
Log Rotation: Configure log rotation with logrotate to manage log file sizes.
Encryption: Encrypt log files or partitions containing logs for enhanced security.
Remote Logging: Forward logs securely using tools like rsyslog to a central server.
Monitoring and Alerts: Use monitoring tools (Nagios, Zabbix) for real-time alerts on suspicious activities.

Using additional tools like Fail2ban, OSSEC, or the ELK Stack, you can significantly enhance your Debian system's logging capabilities while improving security posture through effective log management and analysis.

Example: Installing Additional Packages for Advanced Logging

Fail2ban

Purpose: Protect services from brute-force attacks by monitoring logs for multiple failed login attempts.

$ sudo apt update

Copy

$ sudo apt install fail2ban

Copy

OSSEC

Purpose: Provides host-based intrusion detection by monitoring logs, file integrity, and system configuration changes.

$ sudo apt update

Copy

$ sudo apt install ossec-hids-server ossec-hids-agent

Copy

ELK Stack (Elasticsearch, Logstash, Kibana)

Purpose: Centralized logging and log analysis platform for collecting, searching, and visualizing logs.

# Install Elasticsearch

Copy

$ sudo apt install elasticsearch

Copy

# Install Logstash

Copy

$ sudo apt install logstash

Copy

# Install Kibana

Copy

$ sudo apt install kibana

Copy

Useful Commands for Log Management in Debian

Dmesg

$ sudo dmesg --since -5m

Copy

$ sudo dmesg -w

Copy

$ sudo dmesg | grep iwl

Copy

$ sudo dmesg | grep rtw

Copy

$ sudo dmesg | grep ath

Copy

$ sudo dmesg -T | grep xhci

Copy

Journalctl

$ sudo journalctl -k -b -1

Copy

$ sudo journalctl -p 3 -xb

Copy

$ sudo journalctl -b | grep -i net

Copy

$ sudo journalctl -S -1h00m

Copy

$ sudo journalctl -S today

Copy

$ sudo journalctl -S today -u name.service

Copy

$ sudo journalctl -S "2024-01-01 00:00:00"

Copy

$ sudo journalctl -S "2024-01-01 00:00:00" > ~/journal.txt

Copy

System Logs

$ sudo tail /var/log/syslog

Copy

$ sudo tail -n20 /var/log/syslog

Copy

$ sudo tail -f /var/log/syslog

Copy

$ head /home/$USER/file.txt

Copy

$ head -n20 /home/$USER/file.txt

Copy

Dpkg Logs

$ grep "install" /var/log/dpkg.log

Copy

$ grep "remove" /var/log/dpkg.log

Copy

$ grep "upgrade" /var/log/dpkg.log

Copy

$ grep "status installed" /var/log/dpkg.log

Copy

Terminal output in English

To only run a single command in English, you can write the LANG=C directly in front of the command itself, e.g.

LANG=C sudo apt update

All program output will be in English. You can add a line

export LANG=C

to the end of your ~/.bashrc file and restart the terminal.

9.03 System Boot

https://wiki.debian.org/KernelHeaders

Grub

The simplest way to display your Grub is to press and hold the shift button while booting.

System Boot

∙ Rescue Mode

Debian Live in Rescue Mode

∙ rEFInd (GUI)

Download rEFInd rescue media

https://rodsbooks.com/refind/getting.html

∙ Grub customizer (GUI)

$ sudo apt install grub-customizer

Copy

∙ Restoring grub

$ sudo cp /usr/share/grub/default/grub /etc/default/grub

Copy

$ sudo update-grub

Copy

$ sudo apt -f install

Copy

∙ Menuentries debug

$ sudo ls /boot

Copy

$ sudo ls -l /vmlinuz

Copy

$ sudo ls /boot | grep vmlinuz

Copy

$ sudo dpkg -l | grep grub

Copy

$ sudo dpkg -l | grep linux-image

Copy

$ sudo dpkg -l | grep linux-headers

Copy

$ sudo ls /sys/firmware

Copy

$ sudo cat /etc/default/grub

Copy

$ sudo cat /boot/grub/grub.cfg

Copy

$ sudo cat /boot/grub/grub.cfg | grep menuentry

Copy

$ sudo cat /boot/grub/grub.cfg | grep submenu

Copy

$ sudo cat /etc/grub.d

Copy

$ sudo cat /etc/grub.d/40_custom

Copy

∙ Chroot

$ sudo

Copy

∙ Grub Issues

Removable Medium Boot

$ sudo grub-install /dev/sdX -v --force-extra-removable

Copy

Cryptab

$ sudo nano /boot/grub/grub.cfg

Copy

GRUB_DISABLE_OS_PROBER=false

GRUB_ENABLE_CRYPTODISK=y

$ sudo update-grub

Copy

Remove Old Kernels

An easy way to remove old kernels is to use Synaptic Package Manager to search for "linux-image" and "linux-headers" and remove one except the version you are currently using.

$ sudo apt remove linux-image-6.1.0-15-amd64

Copy

$ sudo apt purge linux-image-6.1.0-15-amd64

Copy

Not recommended

$ dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d'

$ sudo dpkg -l | grep linux-image

Copy

$ sudo dpkg -l | grep linux-headers

Copy

$ sudo ls /boot

Copy

$ sudo ls /lib/modules

Copy

$ sudo rm /boot/*-4.18.0-{15,17}-*

Copy

$ sudo dpkg --configure -a

Copy

$ sudo apt install -f

Copy

$ sudo update-grub

Copy

$ sudo apt autoremove

Copy

$ sudo apt upgrade

Copy

9.04 Display Manager and Monitors

Display Manager and Monitors

https://wiki.archlinux.org/title/Display_manager
https://baeldung.com/linux/display-managers-install-uninstall
https://github.com/ sddm/sddm/releases

Brightness

$ sudo apt install -y brightnessctl

Copy

$ brightnessctl s 100%

Copy

$ brightnessctl s 70%

Copy

$ brightnessctl s 50%

Copy

Environment variables

Qt applications can be scaled with the following environment variables, ote that many applications are hard-coding sizing and font and thus the result on such app may not be as expected.

$ export QT_AUTO_SCREEN_SET_FACTOR=0

Copy

$ export QT_SCALE_FACTOR=1

Copy

$ export QT_FONT_DPI=96

Copy

Monitor and resolution

https://dpi.lv

$ sudo apt install arandr

$ sudo apt install xserver-xorg-input-all

Copy

Debug commands

$ sudo apt install inxi

Copy

$ sudo inxi -G

Copy

$ sudo inxi -Fxxrzc0

Copy

$ sudo xrandr --output eDP-1 --primary

Copy

$ sudo xrandr --output DP2 --auto --left-to DP1

Copy

$ sudo xrandr --output LVDS1 --panning 1920x1080 --scale 1.406x1.406

Copy

$ sudo xrandr --output LVDS1 --panning 1366x768 --scale 1x1

Copy

$ glxinfo | grep "OpenGL"

You can make this change permanent for a specific user by adding this to the startup applications:

$ /usr/bin/xrandr --output LVDS1 --panning 1920x1080 --scale 1.406x1.406

Copy

*Firefox and Thunderbird - Advanced Settings


  layout.css.devPixelsPerPx 0.8

SSDM

Debug commands

$ sudo systemctl status default.target

Copy

$ sudo systemctl status sddm.service

Copy

$ sudo systemctl list-unit-files | grep sddm

Copy

$ sudo ls -la /etc/systemd/system/display-manager.service

Copy

$ cat /proc/cmdline

Copy

$ sudo dpkg-reconfigure sddm

Copy

9.05 Network

https://wiki.ubuntu.com/X/Debugging/WirelessWithoutX
https://wireless.wiki.kernel.org/en/users/drivers/iwlwifi
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/intel
https://intel.com/content/www/us/en/support/articles/000005511/network-and-io/wireless.html
https://wiki.archlinux.org/title/Power_management#USB_autosuspend
https://docs.kernel.org/driver-api/usb/power-management.html

Troubleshooting a network using OSI model, starting from physical to application layer.

First things first:

$ sudo apt install firmware-realtek

Copy or

$ sudo apt install firmware-iwlwifi

Copy or

$ sudo apt install firmware-atheros

Copy

• Network

Debug WiFi by OSI Layers

Application Layer (7)

$ nmcli radio wifi off
$ nmcli radio wifi on
$ sudo ls /etc/NetworkManager/
$ sudo cat /etc/NetworkManager/NetworkManager.conf
$ sudo journalctl -u NetworkManager
$ sudo journalctl -xe NM_CONNECTION=123456 + NM_DEVICE=eth0
$ sudo systemctl status NetworkManager.service
$ sudo systemctl restart NetworkManager.service
$ sudo systemctl status resolvconf.service
$ sudo systemctl restart resolvconf.service
$ sudo systemctl status systemd-resolved.service
$ sudo systemctl restart systemd-resolved.service

Presentation Layer (6)

$ sudo cat /etc/resolv.conf
$ sudo cat /run/resolvconf/resolv.conf
$ sudo cat /lib/systemd/resolv.conf
$ sudo cat /etc/systemd/resolved.conf
$ sudo cat /run/systemd/resolve/stub-resolv.conf
$ sudo cat /var/run/NetworkManager/resolv.conf
$ echo "blacklist conflicting_driver" | sudo tee -a /etc/modprobe.d/blacklist.conf

Network Layer (5)

$ ip address
$ ip route
$ nmcli device show
$ nmcli dev wifi
$ nmcli device show wlan0 | grep IP4.DNS
$ sudo journalctl -b | grep -i net
$ ping -c3 8.8.8.8
$ ping localhost
$ traceroute 8.8.8.8
$ traceroute localhost
$ sudo grep -c dnsmasq /var/log/*
$ sudo grep -c dnsmasq /var/log/syslog
$ sudo dnsmasq --no-daemon --log-queries=extra --log-dhcp --log-debug -C /etc/dnsmasq.conf
$ sudo dhclient -v
$ sudo apt install resolvconf
$ sudo resolvconf --list
$ sudo resolvconf --enable-updates
$ sudo resolvconf -u
$ sudo systemctl status resolvconf.service
$ sudo systemctl restart resolvconf.service
$ sudo apt install systemd-resolved
$ sudo systemctl status systemd-resolved.service
$ sudo systemctl restart systemd-resolved.service

Data Link Layer (4)

$ ip link
$ sudo iwconfig wlan0
$ sudo iwconfig eth0
$ sudo ifconfig wlan0

Physical Layer (3)

$ sudo ifconfig wlan0 down
$ sudo ifconfig wlan0 up
$ sudo ifconfig eth0 down
$ sudo ifconfig eth0 up
$ sudo dmesg | grep iwl
$ sudo dmesg | grep rtw
$ sudo dmesg | grep ath
$ lsusb
$ ls /sys/bus/usb/devices

Data Link Layer (2)

$ ip link
$ sudo iwconfig eth0
$ sudo iwconfig wlan0
$ sudo ifconfig wlan0

Physical Layer (1)

$ sudo ifconfig wlan0 down
$ sudo ifconfig wlan0 up
$ sudo ifconfig eth0 down
$ sudo ifconfig eth0 up
$ sudo dmesg | grep iwl
$ sudo dmesg | grep rtw
$ sudo dmesg | grep ath
$ lsusb
$ ls /sys/bus/usb/devices
$ echo 'on' | sudo tee /sys/bus/usb/devices/1-4/power/control
$ sudo sudo modprobe -rv [WIFIMODULE] && sudo modprobe -v [WIFIMODULE]

*SIOCSIFFLAGS

*RTNETLINK

∙ Getting help

Enable Debugging Features:
- Depending on your Linux distribution, you may need to enable specific debug options. For example, you can enable debug logging for NetworkManager by editing its configuration file:
- sudo nano /etc/NetworkManager/NetworkManager.conf
- Add the following lines under the [logging] section:
- ```
              [logging]
              level=DEBUG
  
```
- Save the file and restart NetworkManager:
- sudo systemctl restart NetworkManager
Send Kernel Logs with Debugging On:
- After enabling debugging features, gather kernel logs and NetworkManager logs:
- sudo dmesg | grep wifi > kernel_logs.txt
- sudo journalctl -u NetworkManager > nm_logs.txt
Be Specific About Hardware and Software:
- Mention the specific hardware details such as the Wi-Fi adapter model, firmware version, Linux distribution, and kernel version.
Describe What Doesn't Work and What You've Tried:
- Clearly explain the issues you are facing with your Wi-Fi, such as connection drops, slow spneeds, or inability to connect. Mention any troubleshooting steps you've already taken, such as restarting the router, checking network settings, or reinstalling drivers.
Include Wireless Captures:
- Use tools like Wireshark to capture wireless traffic for further analysis.
- Install Wireshark if not already installed:
- sudo apt install wireshark
- Capture Wi-Fi packets on your specific interface (e.g., wlan0):
- sudo tshark -i wlan0 -w wifi_capture.pcap
Contacting Support or Forums:
- Once you have gathered all the necessary information, consider reaching out to the official support channels of your Wi-Fi adapter's manufacturer, Linux distribution forums, or community support forums like Stack Exchange or Reddit. Provide all the details you've gathered to get more targeted assistance.

∙ Debug Hardware

$ sudo lspci

Copy

$ sudo lspci -v -k

Copy

$ sudo lspci -v | grep Ethernet

Copy

$ sudo lspci -Knn | grep Net -A2

Copy

$ sudo lsusb

Copy

$ sudo apt install lshw

Copy

$ sudo lshw -C network

Copy

$ sudo apt install inxi

Copy

$ sudo inxi -Fxxz

Copy

*PCI vs USB - Kernel - Integrated chip - Need to investigate

Listing modules

$ sudo lsmod

Copy

$ sudo lsmod | grep iwl

Copy

$ sudo lsmod | grep rtw

Copy

$ sudo lsmod | grep ath

Copy

Module info

$ sudo modinfo rtw_8723d

Copy

Rfkill

$ sudo apt install rfkill

Copy

Commands

$ sudo rfkill list

Copy

$ sudo rfkill unblock wifi

Copy

$ sudo rfkill unblock all

Copy

∙ Deactivating and Activating WiFi Firmware

Deactivating module

$ sudo modprobe -vr rtw_8723d

Copy

$ sudo modprobe -vr rtw_core

Copy

Activating module

$ sudo modprobe -v rtw_core

Copy

$ sudo modprobe -v rtw_8723d

Copy

∙ Power and suspend control

$ sudo ls /etc/NetworkManager/conf.d

Copy

$ sudo touch /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

Copy

$ sudo nano /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

Copy

$ wifi.powersave = 2

Copy or simply

$ sudo sed -i 's/3/2/' /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

<br> pcie_aspm.po
licy=powersasave<br> pcie_aspm.policy=performance<br> usbcore<br>

∙ Wifi stops workin after resume

$ sudo touch /etc/modprobe.d/50-rtw-core.conf

Copy

$ sudo nano /etc/modprobe.d/50-rtw-core.conf

Copy

options rtw_core

$ sudo sudo modprobe -rv [WIFIMODULE] && sudo modprobe -v [WIFIMODULE]

Copy

∙ DNS Issues

Resolvectl: "resolvectl is a multi-call binary. When invoked as "resolvconf" (generally achieved by means of a symbolic link of this name to the resolvectl binary) it is run in a limited resolvconf(8) compatibility mode. It accepts mostly the same arguments and pushes all data into systemd-resolved.service(8), similar to how dns and domain commands operate. Note that systemd-resolved.service is the only supported backend, which is different from other implementations of this command.

Systemd-resolved: "the DNS servers contacted are determined from the global settings in /etc/systemd/resolved.conf, the per-link static settings in /etc/systemd/network/*.network files (in case systemd-networkd.service(8) is used), the per-link dynamic settings received over DHCP, information provided via resolvectl(1), and any DNS server information made available by other system services. See resolved.conf(5) and systemd.network(5) for details about systemd's own configuration files for DNS servers. To improve compatibility, /etc/resolv.conf is read in order to discover configured system DNS servers, but only if it is not a symlink to /run/systemd/resolve/stub-resolv.conf, /usr/lib/systemd/resolv.conf or /run/systemd/resolve/resolv.conf.

/etc/resolv.conf will only be updated with servers added with this command when /etc/resolv.conf is a symlink to /run/systemd/resolve/resolv.conf, and not a static file.

Add a custom DNS

$ sudo nano /etc/systemd/resolved.conf

Copy

[Resolve] DNS=8.8.8.8 1.1.1.1

$ sudo systemctl restart systemd-resolved.service

Copy

$ sudo systemctl status systemd-resolved.service

Copy

$ sudo apt install iproute2

Copy

$ sudo ss -nlup

Copy

$ sudo ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

Copy

  • Syntax checks 
   $ sudo dnsmasq --test
• Print errors 
$ sudo grep -c dnsmasq /var/log/* 
$ sudo grep -c dnsmasq /var/log/syslog 
$ sudo dnsmasq --no-daemon --log-queries=extra --log-dhcp --log-debug -C /etc/dnsmasq.conf 
$ sudo dhclient -v

Disabling NetworkManager's own dnsmasq

👷🛠️UNDER CONSTRUCTION🚧🏗

  $ sudo nano /etc/NetworkManager/NetworkManager.conf #dns=dnsmasq 
 
  $ sudo restart network-manager

Disabling NetworkManager's IPV6

$ sudo

Copy

$ sudo

Copy

NetworkManager dnsmasq (CLI)

DNS requests are directed to VPN-supplied DNS servers without any manipulations with dnsmasq, up/down/dispatch helper scripts.

mcli -p connection modify MY_VPN_CONNECTION ipv4.never-default no nmcli -p connection modify MY_VPN_CONNECTION ipv4.ignore-auto-dns no nmcli -p connection modify MY_VPN_CONNECTION ipv4.dns-priority -42

*Using OpenVPN through NetworkManager (GUI) allows users to disable the connection.

∙ DHCP Issues

$ sudo apt install tcpdump

Copy

$ sudo apt install nmap

Copy

$ sudo apt install wireshark

Copy

9.06 USB Devices

https://wiki.ubuntu.com/Kernel/Debugging/USB
https://wiki.archlinux.org/title/Power_management#USB_autosuspend
https://docs.kernel.org/driver-api/usb/power-management.html
https://wiki.debian.org/HowToIdentifyADevice/USB
https://wiki.debian.org/HowToIdentifyADevice/PCI
https://kernel.org/doc/html/latest/usb/index.html
https://kernel.org/doc/html/v4.16/driver-api/usb/power-management.html
UAS Issues - https://forums.raspberrypi.com/viewtopic.php?t=245931
Bug - xhci_hcd WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state - https://bugzilla.kernel.org/show_bug.cgi?id=202541
Bug - CPU hard lockup related to xhci/dma - https://bugzilla.kernel.org/show_bug.cgi?id=217242
Bug - Debootstrap is very slow. Please use eatmydata to fix this. - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700633

Tip: If you are transferring large amounts of data via a problematic USB, use grsync as a manager.

USB debug

$ lsusb -t

Copy

$ lsusb -v

Copy

$ lsusb -v | grep 1111

Copy

$ usb-devices

Copy

$ sudo dmesg -w

Copy

$ sudo dmesg -T | grep xhci

Copy

$ sudo lspci -v | grep xhci

Copy

$ sudo grep -i xhci /boot/config-$(uname -r)

Copy

See usb device getting plugged in

$ sudo dmesg -wH

Copy

Firmware

$ sudo modinfo xhci_hcd

Copy

$ sudo modinfo ehci_hcd

Copy

$ sudo modinfo btusb

Copy

$ sudo modprobe -v ohci-pci

Copy

$ sudo modprobe -v ehci-hcd

Copy

$ sudo modprobe -v xhci-hcd

Copy

$ sudo modprobe -v xhci-pci

Copy

You may try to force your system to use USB 2.0 insted of USB 1.1:

$ sudo modprobe -vr ohci-pci

Copy

$ sudo modprobe -v ehci-hcd

Copy

Deactivate

$ sudo modprobe -vr xhci-hcd

Copy

$ sudo modprobe -vr xhci-pci

Copy

Activate

$ sudo modprobe -v xhci-hcd

Copy

$ sudo modprobe -v xhci-pci

Copy

Power and suspend control

$ lsusb

Copy

$ ls /sys/bus/usb/devices

Copy

$ echo 'on' | sudo tee /sys/bus/usb/devices/1-4/power/control

$ sudo cat /etc/tlp.conf

Copy

$ grep . /sys/bus/usb/devices/*/power/autosuspend

$ grep . /sys/bus/usb/devices/*/power/autosuspend | awk -F: '{print $1}' | xargs -I{} sudo sh -c 'echo "-1" > {}'

$ grep . /sys/bus/usb/devices/*/power/control

$ grep . /sys/bus/usb/devices/*/power/control | awk -F: '{print $1}' | xargs -I{} sudo sh -c 'echo "on" > {}'

$ grep . /sys/bus/usb/devices/*/power/wakeup

$ grep . /sys/bus/usb/devices/*/power/wakeup | awk -F: '{print $1}' | xargs -I{} sudo sh -c 'echo "disabled" > {}'

If your keyboard, mouse, or other devices stop working sometimes, power management might be affecting you. Execute this command to check:

$ cat /sys/module/usbcore/parameters/autosuspend

You are affected if you get back a "2". To disable it, simply write a -1 to that file as root:


   $ sudo su

# echo -1 > /sys/module/usbcore/parameters/autosuspend

To make the change permanent, edit the GRUB configuration:


   $ sudo nano /etc/default/grub

Add usbcore.autosuspend=-1 to the end of the command in GRUB_CMDLINE_LINUX_DEFAULT :

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash usbcore.autosuspend=-1"

Then update GRUB:


   $ sudo update-grub

Alternatively, you can use this one-line command to modify the configurations:


   $ sudo sudo sed -i.bak '/^GRUB_CMDLINE_LINUX_DEFAULT/ s/"$ usbcore.autosuspend=-1"/' /etc/default/grub && sudo update-grub

Verify:


   $ sudo cat /etc/default/grub

UAS Issues

https://forums.raspberrypi.com/viewtopic.php?t=245931

Symptoms of a misbehaving UAS device

• Slow transference
• Frequent disconnects-reconnects
• Dmesg reports errors relating to a UAS device.

Solving

Plug in the USB device(s) and run the command dmesg

$ sudo dmesg --since -2m

Copy

Take note of the VID (idVendor) and PID (idProduct) of your USB device(s)

(...) [ 4906.696463] usb 2-1: New USB device found, idVendor=1111, idProduct=2222, bcdDevice=c3.33 (...)

Add the quirks to /boot/cmdline.txt

$ sudo nano /boot/cmdline.txt

Copy

Add the text with your idVendor and idProduct, respectively.

usb-storage.quirks=1111:2222:u

usb-storage.quirks=4444:5555:u,6666:7777:u.,8888:9999:u

Reboot

$ sudo reboot

Copy

Check

$ sudo dmesg | grep usb-storage

Copy

udisks2.service

$ sudo systemctl status udisks2.service

Copy

$ sudo apt install libblockdev-crypto2 libblockdev-mdraid2 --no-install-recommends

Copy

intel_iommu=off

legacy USB

Into bios, have usb 3.0 turned on, an any other options turned on, but turn off legacy usb option.But think that after legacy USB disabling, in some cases, you could lost ability to enter your BIOS, if you have USB keyboard, because your keyboard will not work at the moment when you need to press DEL or F2 or whatever.

Mouse

https://github.com/sriemer/fix-linux-mouse

9.07 Bluetooth

Bluetooth

https://wiki.archlinux.org/title/Bluetooth
https://wiki.archlinux.org/title/Bluetooth#Troubleshooting
https://wiki.debian.org/BluetoothUser
https://wiki.debian.org/Bluetooth/Alsa
https://github.com/Arkq/bluez-alsa

First things first:

$ sudo apt install firmware-realtek

Copy or

$ sudo apt install firmware-iwlwifi

Copy or

$ sudo apt install firmware-atheros

Copy

Audio compatibility

$ sudo apt install pulseaudio-module-bluetooth

Copy or

$ sudo apt install bluez-alsa-utils

Copy

Debug

$ sudo service bluetooth status

Copy

$ hciconfig -a

Copy

$ hciconfig hci0

Copy

$ sudo dmesg | grep Bluetooth

Copy

$ sudo dmesg | grep iwl

Copy

$ sudo dmesg | grep rtl

Copy

$ sudo dmesg | grep ath

Copy

Connecting Bluetooth

  Commands
  • Connecting bluetooth manualy with Bluez
  $ bluetoothctl [bluetooth]# scan on [bluetooth]# pair xx:xx:xx [bluetooth]# connect xx:xx:xx [bluetooth]# trust xx:xx:xx [bluetooth]# block yy:yy:yy

Bluetooth Managment

  Commands
  • Bluetooth managment
  $ btmgmt [btmgmt]# info

Bluetooth Audio ALSA Backend (bluealsa)

https://github.com/arkq/bluez-alsa/wiki/Bluetooth-Pairing-And-Connecting

$ sudo apt install bluez-alsa-utils

Copy

Mouse

https://github.com/sriemer/fix-linux-mouse

9.08 Sound

https://wiki.ubuntu.com/DebuggingSoundProblems
https://wiki.archlinux.org/title/Advanced_Linux_Sound_Architecture
https://wiki.archlinux.org/title/PulseAudio/Troubleshooting
https://thesofproject.github.io/latest/getting_started/intel_debug/suggestions.html
https://wiki.debian.org/Sound
https://wiki.debian.org/SoundConfiguration
https://alsa-project.org
https://github.com/thesofproject/sof/issues
https://github.com/ thesofproject

First things first:

$ sudo apt install firmware-sof-signed

Copy

$ sudo apt install firmware-intel-sound

Copy

Debug commands

List of PLAYBACK Hardware Devices

$ aplay -l

Copy

$ lspci |grep -i audio

Copy

$ cat /proc/asound/cards

Copy

$ cat /proc/asound/card*/id

Copy

List of CAPTURE Hardware Devices

$ arecord -l

Copy

Testing

$ speaker-test

Copy

Other debug commands

$ cat /proc/asound/cards

Copy

$ cat /proc/asound/modules

Copy

$ lspci -v | grep Audio

Copy

$ lsusb

Copy

$ lsmod | grep snd

Copy

$ aplay -lL

Copy

$ amixer

Copy

$ amixer -c0

Copy

$ sudo dpkg-reconfigure linux-sound-base

Copy

$ sudo alsa-info

Copy

ALSA

The "Advanced Linux Sound Architecture" (ALSA) is a part of the Linux kernel. PulseAudio is a sound server that sits between ALSA and user applications, aiming to provide easy automatic sound configuration for users. PulseAudio controls underlying ALSA-level volume controls.

$ sudo apt install alsa-utils

Copy

$ alsamixer

Copy

$ amixer -c 0 set Master 100%

Copy

$ amixer -c 1 set Speaker 50%

Copy

$ amixer -c 1 set Speaker 3db

Copy

$ amixer -c 1 set Speaker 2db+

Copy

$ amixer -c 0 set Mic unmute

Copy

$ sudo nano /etc/pulse/default.pa

Copy

PulseAudio Volume Control

Allows you to control both the volume of hardware devices and of each playback stream separately. It also allows you to redirect a playback stream to another output device.

$ sudo apt install pavucontrol

Copy

Advanced solution

Install the SOF firmware binaries from the source: https://github.com/thesofproject/sof-bin

Clone the repository:

$ git clone https://github.com/thesofproject/sof-bin.git

Copy

Change to directory:

$ cd sof-bin

Copy

Follow: https://github.com/thesofproject/sof-bin#install-process-with-installsh

$ sudo mv /lib/firmware/intel/sof* some_backup_location/

Copy

$ sudo mv /usr/local/bin/sof-* some_backup_location/ # optional

Copy

$ sudo ./install.sh v2.2.x/v2.2

Copy

Reboot

$ sudo reboot

Copy

"Apollo Lake" platform

https://thesofproject.github.io/latest/getting_started/intel_debug/suggestions.html#es8336-support

Check your kernel configuration, typically available as a /boot/config-*. These options below will allow you to use Sound Open Firmware, start with:

$ grep SND_SOC_INTEL_APL /boot/config-*

Copy

For your system you could select:

ALSA for SoC audio support (CONFIG_SND_SOC=m) Intel ASoC SST drivers (CONFIG_SND_SOC_INTEL_SST_TOPLEVEL=y) and the appropriate platform option, which in your case would be "Broxton/ApolloLake platforms" (CONFIG_SND_SOC_INTEL_APL=m)

Others options:

Sound Open Firmware support (CONFIG_SND_SOC_SOF_TOPLEVEL=y) SOF support for Intel Audio DSPs (CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL=y) SOF PCI enumeration support (CONFIG_SND_SOC_SOF_PCI=m) SOF support for Apollolake (CONFIG_SND_SOC_SOF_APOLLOLAKE_SUPPORT=m) SOF support for HDA Links (HDA/HDMI) (CONFIG_SND_SOC_SOF_HDA_LINK=y) SOF support for HDAudio codecs (CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC=y)

9.09 Printers

Printers

https://wiki.debian.org/SystemPrinting
https://wiki.ubuntu .com/DebuggingPrintingProblems

∙ HP

https://developers.hp.com/hp-linux-imaging-and-printing/install/manual/distros/debian
https://wiki.debian.org/InstallingDebianOn/HP
https://developers.hp.com/hp-linux-imaging-and-printing

$ sudo apt install hplip

Copy

∙ EPSON

https://epson.com/Support/wa00821

$ sudo apt install

Copy

∙ Wireless Firewall Ports

9.10 Keyboard

https://debian.org/doc/manuals/debian-reference/ch08.en.html#_the_keyboard_input
https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_002
man xbindkeys
man xdotool

Keyboard Debug

$ locale

Copy

$ locale -a

Copy

Keyboard Tip

The keysym to trigger the context menu is called Menu. It can be bound to an arbitrary keycode like this:


  $ xmodmap -e 'keycode 68 = Menu'

Can be manually triggered from the command line like:


  $ xdotool key Menu

And to find out the keycode it is currently bound to one can do:


  $ xmodmap -pke | grep Menu

To figure out what keycode a key emits one can use:


  $ xev

Reset Layout

$ sudo dpkg-reconfigure keyboard-configuration

Copy

$ sudo service keyboard-setup restart

Copy

$ sudo update-initramfs -u

Copy

Accents not working

Can't type accented letters in certain programs.

Assign Home and End to Fn + Arrows

https://superuser.com/questions/428945/defining-keyboard-shortcuts-involving-the-fn-key
https://superuser.com/questions/1069211/assign-home-and-end-to-fnarrows

Remap Apple Magic Keyboard

https://github.com/dongjinleekr/hid-apple-numberless-ko
https://github.com/isakhauge/nor-apple-keyboard-xmodmap

9.11 Locales and Time/Date

https://man7.org/linux/man-pages/man1/locale.1.html
https://linuxfromscratch.org/lfs/view/stable-systemd/chapter09/locale.html
https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_002

Locales

$ locale

Copy

$ sudo apt reinstall locales

Copy

$ sudo locale-gen

Copy

$ sudo locale-gen en_US.UTF-8

Copy

$ sudo dpkg-reconfigure locales

Copy

*Logoff

If locale-gen had not results

$ locale

Copy

$ export LANGUAGE=en_US.UTF-8

Copy

$ export LC_ALL=en_US.UTF-8

Copy

$ export LANG=en_US.UTF-8

Copy

$ export LC_CTYPE=en_US.UTF-8

Copy

Even no results

$ export LC_ALL="C.UTF-8"

$ sudo dpkg-reconfigure locales

Copy

Even no results

In /etc/locale.gen, uncommenting the line: en_US.UTF-8 UTF-8 then running: locale-gen

Switch terminal command output

UNDER TEST! NOT FULL TESTED! https://manpages.ubuntu.com/manpages/trusty/man7/locale.7.html

• Defitenelly change output language from native language to english
     $ env
     $ env | egrep -e 'LC_ALL|LANG'
     $ cat ~/.bashrc
     $ cat ~/.bash_profile
     $ LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LANGUAGE=en_US.UTF-8 $SHELL

Bash function for terminal

Switching between DE and EN locales. Put it in your ~/.bashrc (or ~/.bash_profile).

Call it with _configure_locale EN to switch to English.

function _configure_locale() { # [profile] local profile=${1:-EN} case ${profile} in DE|DE_DE|de_DE) LC_ALL="de_DE.UTF-8" LANG="de_DE.UTF-8" LANGUAGE="de_DE:de:en_US:en" ;; EN|EN_US|en|en_US) LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8" LANGUAGE="en_US:en" ;; *) echo "ALERT" "${FUNCNAME}: unknown profile '${profile}'" ;; esac LC_PAPER="de_DE.UTF-8"; # independent from locale LESSCHARSET="utf-8"; # independent from locale MM_CHARSET="utf-8" # independent from locale echo "locale settings" "${LANG}"; export LC_ALL LANG LANGUAGE LC_PAPER LESSCHARSET MM_CHARSET }

Time and Date

https://wiki.archlinux.org/title/Installation_guide#Time
https://wiki.archlinux.org/title/Installation_guide#Hardware_clock

Control the system time and date

Display the manual page for timedatectl, a command-line tool that helps manage system time and date settings.

$ man timedatectl

Copy

Show the current system time, date, and related settings.

$ timedatectl status

Copy

Select your time zone from a list of options.

$ tzselect

Copy

How to Reconfigure time and date

Reconfigure the time zone data by running the package configuration tool.

$ sudo dpkg-reconfigure tzdata

Copy

How to Reconfigure BIOS time to UTC (Coordinated Universal Time)

Set the hardware clock to use Coordinated Universal Time (UTC).

$ sudo timedatectl set-local-rtc 0

Copy

Reconfigure BIOS time with NTP time

Synchronize the BIOS (hardware clock) with the system clock using the NTP (Network Time Protocol) time.

Update the BIOS clock to match the system clock, which has been synchronized with the NTP server.

$ sudo hwclock --systohc

Copy

Adjtime

adjtime is a system call and a command in Unix-like operating systems used to gradually adjust the system clock. It corrects small discrepancies in the system time without causing abrupt changes, ensuring smooth and continuous timekeeping. This is particularly useful for time synchronization.

System Call

As a system call, adjtime allows programs to specify an amount of time to be added or subtracted from the system clock gradually, preventing sudden jumps in time.

Command

As a command, adjtime can apply these adjustments directly. The hwclock utility interacts with /etc/adjtime to maintain the hardware clock in sync with the system clock.

/etc/adjtime File

This file is used by the hwclock command to track the drift of the hardware clock. It contains three lines:

The last time hwclock was used to set the system clock.
The time difference between the hardware clock and the system clock at the last hwclock run.
The adjustment mode (UTC or LOCAL).

Example Usage

When you run hwclock --systohc, it updates the hardware clock to match the system clock and records the difference in /etc/adjtime. This helps in maintaining accurate time across reboots.

Benefits

Smooth Time Adjustment: Gradually adjusts the system time, avoiding abrupt changes.
Accurate Timekeeping: Helps maintain synchronization between the hardware clock and the system clock.
Time Drift Correction: Corrects small drifts in the hardware clock, improving long-term accuracy.

Example Command

This command uses the information in /etc/adjtime to correct manually the hardware clock drift.

$ sudo hwclock --adjust

Copy

NTP (Network Time Protocol)

Install ntpdate, synchronize the system clock with a network time server, and reconfigure the time zone data.

$ sudo apt install ntpdate && sudo ntpdate in.pool.ntp.org && sudo dpkg-reconfigure tzdata

Copy

∙ Coordinated Universal Time - UTC-00:00

Systems synchronized to a common time source (such as an NTP server using UTC) are less likely to experience issues related to time drift, which can affect security protocols and processes.

Set the system time zone to Coordinated Universal Time (UTC).

$ sudo timedatectl set-timezone UTC

Copy

∙ Greenwich Mean Time - GMT

Set the system time zone to Greenwich Mean Time (GMT).

$ sudo timedatectl set-timezone GMT

Copy

Configuring Windows 10 and Linux for Consistent BIOS Time in Multiboot

Configure Windows 10 to Use UTC

Press Win + R, type regedit, and hit Enter to open the Registry Editor.

Navigate to the following path:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation

Right-click in the right pane and select New > DWORD (32-bit) Value.
Name the new value RealTimeIsUniversal.
Double-click RealTimeIsUniversal and set its value to 1.
Close the Registry Editor and restart your computer.

Configure Linux to Use Local Time

Open a terminal and run:
```
$ sudo timedatectl set-local-rtc 1
```
Copy
Configure the system to use local time for the hardware clock.
Ensure your time is correctly updated using NTP:
```
$ sudo apt install ntpdate
```
Copy
Install ntpdate for synchronizing the system clock with a network time server.
```
$ sudo ntpdate pool.ntp.org
```
Copy
Synchronize the system clock with a specified network time server.

9.12 Torrenting

General debug

1. Resetting qBittorrent / trying another torrent client 2. Trying another device on same network 3. Trying ethernet instead of Wi-Fi 4. Trying downloading to external storage rather than SSD 5. Stopping all running apps/services in background 6. Running speed tests to see if it also happens outside torrenting

Download speed fluctuations and near zero uploading, speed spikes

Possible causes

1. Your external disk case, awful flash drive or internal SSD just not being able to handle the simultaneous writes and reads. 2. Your VPN limiting you bandwidth. 3. Your ISP throttling. Torrent traffic throttled even through a VPN.

Possible solutions

Disk Benchmarks Test

$ sudo apt install gnome-disk-utility

Copy

For your ISP throttle, try your VPN Obfuscated Servers or P2P Servers.

Do a Leak Test

∙ DNSLeakTest.com (run the "Extended test")
∙ IPLeak.net

curl ipleak.net/json/

curl ipinfo.io

Alternativelly, begin to VPN+Torrent on PC, wait until throttling begins. Try to play back a 1080p video on your smartphone, try to choose one where you can see the quality difference (for example one with many text elements such as computer hardware benchmarks). A 1080p video nneeds about 1-2Mbps (250KB/s) bandwidth.

Instead you can try to download something on the phone. This is to find out if it's an issue VPN server/software or with your ISP line as a whole. It's possible for your ISP to only throttle the connection to the VPN server and leave the rest alone.

Problem 2

$ sudo

Copy

9.13 Disks

For further reading on SSD optimization, visit:

Disk Managers

GParted

$ sudo apt install gparted

Copy

Gnome Disk - "Disks"

$ sudo apt install gnome-disk-utility

Copy

Disk Manager with LVM Support

KDE Partition Manager

$ sudo apt install partitionmanager

Copy

Disk Debug

Logical and Non-Destructive Methods

Kernel message buffer for USB:
This command filters the kernel message buffer for lines containing 'xhci', which relates to USB 3.x host controller interfaces.
```
$ dmesg -T | grep xhci
```
USB device tree view:
This command displays detailed information about USB devices connected to the system in a tree view.
```
$ lsusb -tv
```
List partition tables:
This command lists the partition tables of the specified device (e.g., /dev/sdb). Useful for identifying partition layout.
```
$ sudo fdisk -l /dev/sdb
```
EXT - Non-interactive filesystem check and repair:
This command performs a non-interactive filesystem check and repair on the specified ext filesystem partition.
```
$ sudo e2fsck -p /dev/sde1
```
EXT - Check for bad blocks and update list:
This command checks for bad blocks on the specified ext filesystem partition and updates the bad block list.
```
$ sudo e2fsck -c /dev/sde1
```
EXT - Load and update bad block list:
This command loads a list of bad blocks from a file and updates the filesystem on the specified partition.
```
$ sudo e2fsck -l /dev/sdb1 /badblock/file
```
SMARTMONTOOLS:
Install smartmontools to check the health of HDDs, SSDs, and NVMe drives.
```
$ sudo apt install smartmontools
```
Copy
Commands for smartmontools:
```
$ sudo smartctl -a /dev/sda
```
Displays all SMART information for the specified disk.
```
$ sudo smartctl -t short /dev/sda
```
Performs a short self-test on the specified disk.
```
$ sudo smartctl -t long /dev/sda
```
Performs a long self-test on the specified disk.
HDPARM:
Install hdparm to view and test HDD performance.
```
$ sudo apt install hdparm
```
Copy
Commands for hdparm:
```
$ sudo hdparm -I /dev/sda
```
Displays detailed information about the specified disk.
```
$ sudo hdparm -t /dev/sda
```
Performs a read speed test on the specified HDD.
```
$ sudo hdparm -T /dev/sda
```
Performs a cached read speed test on the specified HDD.
KDiskMark:
Install KDiskMark to benchmark disk performance for HDDs, SSDs, and NVMe drives.
```
$ sudo apt install -y kdiskmark
```
Copy

Destructive/Risk Methods

DEBUGFS:
Use with caution: debugfs is a filesystem debugger that can manipulate the filesystem directly.
```
$ sudo debugfs
```
Copy
DD Command (Data Destroyer):
These commands overwrite the entire disk with either zeros or random data, destroying all data on the disk.
```
$ sudo dd if=/dev/zero of=/dev/sda bs=1M
```
Copy
```
$ sudo dd if=/dev/urandom of=/dev/sda bs=1M
```
Copy

References

9.14 Hardware

Hardware Utilities

$ sudo apt install lshw

Copy

$ sudo apt install inxi

Copy

$ sudo apt install cpu-x

Copy

$ sudo apt install hardinfo

Copy

$ sudo apt install s-tui stress

Copy

10. OTHERS

10.01 Open Source

Foundations and Organizations

Linux and Debian

Builds and Reproducibility

Freedesktop.org and Wayland

Standards and Security

Training and Learning

10.02 Shell e Others

Shell e others

Linux Blogs

Security

Self-hosting

Eletronics

10.03 YouTube Links

YouTube Links

Digital Forensics and Incident Response:

Security Conferences:

General Tech Conferences:

Miscellaneous Tech and Cybersecurity:

10.04 Libraries

Libraries

10.05 Others

Misc

https://standards.iteh.ai

Counterfeits/Piracy

Back to Top ⬆

Made with ♥

RENANZG/My-Debian-GNU-Linux

About

Topics

Resources

Stars

Watchers

Forks

Languages