TIL: Very useful Linux/Unix commands

Here is a list of useful unix commands or code parts. Who does not know it? You have a problem and looking for a solution where you find at stack overflow or similar pages? Here I collect all the commands that I have encountered over time or whose switch I simply can not remember (or want).

  • How do I find all files containing specific text?
    grep -rnw '/path/to/somewhere/' -e 'pattern'
  • How i change the default file permissions (mask that controls file permissions)
    umask
  • Untar (unzip) file/folder
    tar -zxvf archive.tar.gz
  • Tar (zip) file/folders
    tar -cvzf archive.tar.gz file1 file2
  • Copy files via rsync from one host to another
    rsync -avz [USER@]HOST:SOURCE [USER@]HOST:DEST
    rsync -avz [USER@]HOST:SOURCE rsync://[USER@]HOST[:PORT]/DEST
    rsync -avz -e "ssh -p 12345" LOCAL/SOURCE [USER@]HOST:DEST
  • Using rsync with sudo on the destination machine
    1. Find out the path to rsync: which rsync
    2. Edit the /etc/sudoers file: sudo visudo
    3. Add the line <username> ALL=NOPASSWD:<path to rsync>, where username is the login name of the user that rsync will use to log on. That user must be able to use sudo

Then, on the source machine, specify that sudo rsync shall be used:

rsync -avz --rsync-path="sudo rsync" SOURCE [USER@]HOST:DEST
  • Preserve SSH_AUTH_SOCK (Environment Variables) When Using sudo

    sudo --preserve-env=SSH_AUTH_SOCK -s
  • nslookup missing? Install dig

    sudo apt-get install dnsutils
  • find without "Permission denied"

    find / -name 'filename.ext' 2>&1 | grep -v "Permission denied"
  • flush dns cache

    sudo systemd-resolve --flush-caches
  • show open ports

    netstat -tulpn
  • Directory size

    du -sh /var
    du -shc /var/*
    du -h --max-depth=1 /var
    du -sh /var/lib/docker/containers/*/*.log
  • Search multiple PDF files for a "needle"

    pdfgrep -i needle haystack*.pdf
  • Show hidden files with ls

    ls -lar
  • Redirect STDOUT and STDERR to a file

    nice-command > out.txt 2>&1
  • Installs your SSH public key to a remote host

    sh-copy-id 'user@remotehost'
  • A command-line system information tool

    neofetch
  • Show disk usage, folder size, items per folder, find big directorys, ... ncdu

    gdu
  • Fancy resource monitor

    btop
  • Display disk activity

    iotop
  • Display network activity

    iftop or iptraf
  • Cleanup Docker

    docker system prune --help
  • Find and repair disk errors on ext (ext2, ext3 and ext4) filesystems

    sudo e2fsck -f </dev/sda2> 
  • Forward TCP/UDP ports with socat

    socat TCP-LISTEN:8080,fork,reuseaddr TCP:homeserver.local:8080
  • Traceroute and ping in one, visual console tool

    mtr <host>
  • tl;dr - simplified man pages

    tldr
  • Extend partition to fill empty space (from cloud-untils)

    growpart /dev/sda 1
  • traceroute implementation using TCP packets instead of UDP/ICMP

    tcptraceroute
  • List all commands that require sudo

    sudo -l
  • Print info about the Linux distribution (name, version, ID, etc.)

    cat /etc/os-release
  • Pipe tcpdump directly to Wireshark on Windows when sshdump or port forwarding not working

    ssh user@hostname "sudo tcpdump -i eth0 -U -s0 -w - not port 22" | "C:\Program Files\Wireshark\Wireshark.exe" -k -i -

Run command in background on a Synology NAS with nohup

To run a programm  that doesn't quit if you close the ssh session use nohup (no hangup). Attention, you have to run it as root for it to work!

admin@DiskStation:~$ sudo su
Password:
ash-4.3# nohup <command> &

maybe:
ash-4.3# nohup cat /dev/zero | split -b 4095m - /volumeUSB2/usbshare/zeros -d --additional-suffix=.file &

PowerCLI error after vCenter Server upgrade

After upgrading to vSphere 6.7, a PowerCLI script aborts with this error message:

The vCenter Server is unable to decrypt passwords stored in the customization specification.

To resolve the issue, retype the password in the VMcustomization specifications (under Policies and Profiles). Edit the customization specifications and retype the password under the following two preference points:
- Administrator password
- Workgroup or domain

Write zeros to a hard drive – Wipe/Erase unused or free space

Windows: Format and write zeros to every sector of the drive.
format <Driveletter> /fs:NTFS /p:0

Windows: Format and write zeros to every sector of the drive. After the first pass write random numbers.
format <Driveletter> /fs:NTFS /p:2

Windows: Fill free space.
cipher /w:<Driveletter>[:\foldername]</span>

Unix: Wipe full drive.
dd if=/dev/zero of=/dev/<block device> bs=<block size 1M, 32M, etc.> status=progress

Unix: Fill free space.
dd if=/dev/zero of=/path/to/drive/zeros.file status=progress

Unix: Fill free space on FAT32 drive (4GB limit per file).
cat /dev/zero | split -b 2000m - zero -d --additional-suffix=.file

To run the process in background see Run command in background on a Synology NAS with nohup

How to configure RPC dynamic port allocation to work with firewalls

Windows XP use per default a dynamic port range from 1024 to 5000 for RPC/WMI/DCOM. To set up a fixed Port or Range use this settings:

  1. regedit: HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc
  2. Add key "Internet"
  3. Add this three values:

Ports: REG_MULTI_SZ: 5000-5100
PortsInternetAvailable: REG_SZ: Y
UseInternetPorts: REG_SZ: Y

  1. run winmgmt -standalonehost
  2. net stop winmgmt
  3. net start winmgmt
  4. Reboot

[via]https://support.microsoft.com/en-us/help/154596/how-to-configure-rpc-dynamic-port-allocation-to-work-with-firewalls[/via]
[via]https://msdn.microsoft.com/en-us/library/bb219447%28v=vs.85%29.aspx[/via]

Error: “A later version of Microsoft .NET Framework XY Client Profile is already installed”

Looking at the MSI installer logs (Temp-Dir) and finding this line: NEWERVERSIONDETECTED = {}

And running this in a command line: msiexec /uninstall {}

[via]https://social.msdn.microsoft.com/Forums/vstudio/en-US/a8ceb77b-f625-4259-94de-4ae5157ba98c/net-framework-4-reinstallation-failure?forum=netfxsetup[/via]