Debian cheatsheet

Table of Contents


This has grown into a very useful list of things that make my life easier. Divided into sections for clarity. Mostly applicable to Ubuntu as well.

Nifty shortcuts, one-liners and useful programs

  • netcat for file transfer receiver: nc -l -p 1234 > somefile.png
    sender: nc -w 3 192.168.178.20 1234 < somefile.png
  • ssh port forwarding ssh -R 80:localhost:8080 user@server.com -i key.pem
  • multiple x sessions exec startx -- :1 vt8
  • get random SALT openssl rand -base64 32
  • use imagemagick to resize pictures convert <infile> -resize 100x100 [<outfile>]
  • find and remove “~” files from emacs etc. find . -name*\~ | xargs rm
  • use quvi to download videos etc quvi --exec "wget -O <name.webm> %u" <link>
    or just use youtube-dl
  • count words (or lines) wc [-l]
  • print all file types in a directory find . -type f -name "*.*" | awk -F. '{print $NF}' | sort -u
    or just print number of files of one filetpye: find . -name "*.mp3" -type f | wc -l
  • count number of files find <dir> -type f | wc -l
  • generate random number shuf -i 2000-65000 -n 1
  • recover data from btrfs partitions from superuser
    run strings /dev/sda1 | less and then use / to search for the strings that could be in that file. delimit the start of it with “m” then move to the end and use |cat >file to save it to a file
    (note: worked on text for me, binary could be difficult)
  • tre-agrep
    cat file.txt | tre-agrep -i(ignorecase) -s(show match cost) -3(max cost of 3) 'Fuzzy match'
    • -D: cost of missing characters
    • -I: cost of extra characters
    • -S: cost of substitutions(should be x2 since deletion and insertion are 2 operations)
  • load testing use siege --concurrent=100

Pdf handling

  • use pdftk to split a document into multiple pages pdftk A=input.pdf cat A1-100 output output-1-100.pdf

  • use pdftk to join multiple pdfs
    pdftk input1.pdf input2.pdf cat output output.pdf

  • use pdfjam to make presentations printable
    pdf90 inputs.pdf pdfjam Inputs_rotated.pdf --nup 2x1 --landscape --outfile Outname.pdf pdf270 Outname.pdf

  • convenient fish function

    :::fish
    function pp -d 'Make presentations printable'
        pdf90 $argv
        set -x pdf_file (echo $argv | cut -f 1 -d '.')
        pdfjam (echo $pdf_file)-rotated90.pdf --nup 2x1 \
        --landscape --outfile print-(echo $pdf_file).pdf
        pdf270 print-(echo $pdf_file).pdf
        rm print-(echo $pdf_file).pdf
        mv print-(echo $pdf_file)-rotated270.pdf print-(echo $pdf_file).pdf
        rm (echo $pdf_file)-rotated90.pdf
    end
    
  • k2pdfopt
    -om 0.3 for margins, -n for disabling ocr and bitmap generation

  • removing margins
    use pdfcrop to remove whitespace automatically
    use --margins '5 10 20 30' to remove margins in pts(left, top, right, bottom), use negative values (-50 -50 -50 -50) for actually cropping away

  • generate blank page
    echo "" | ps2pdf -sPAPERSIZE=a4 - blankpage.pdf

Misc

(Difficult-to-remember commands or usage of programs, accomplishing very specific tasks)

  • tar tar xvzf file to extract file.tar.gz, tar cvzf file.tar.gz files_to_include to create

  • find find . -iname "*thing*", case insensitive

  • multiple keyboard locales setxkbmap -option grp:switch,grp:alt_shift_toggle,grp_led:scroll us,de

  • encrypt harddrive (from hermann-uwe.de)

    :::bash
    badblocks -c 10240 -s -w -t random -v /dev/sdX
    # create partition using fdisk
    cryptsetup --verbose --verify-passphrase luksFormat /dev/sdX1
    cryptsetup luksOpen /dev/sdX1 name
    mkfs.ext3 -j -m 1 -O dir_index,filetype,sparse_super /dev/mapper/name
    # mount it
    # write to it
    # unmount it
    cryptsetup luksClose /dev/mapper/name
    
  • invert bash wildcards
    shopt (-s)(-u) extglob
    ls !(*.zip)

  • the sudo tee duo sh unprivileged-command | sudo tee output-file >/dev/null

  • sed instead of head sed 11q file is equal to head file
    sed '3q;d' file to get 3rd line of file

  • bridge terminal and clipboard
    echo test | xclip -selection c - put into clipboard
    xclip -selection clipboard -o - print out clipboard contents

  • use notify-send sanely
    use notify-send 'Heading' 'Body text' --icon=dialog-information with --hint int:transient:1 for ephemeral notifications that do not clutter the message view

  • qemu
    qemu-img create disk.img 10G
    qemu -hda disk.img -boot d -cdrom distro.iso -m 1024

  • download a whole webpage
    wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains domain.com domain.com

  • reinstall all python packages
    sudo dpkg --get-selections | sed 's:install$::' | sed -e 's/^[ \t]*//' -e 's/[ \t]*$//' | grep -v 'python3' | grep python sudo dpkg --get-selections | sed 's:install$::' | sed -e 's/^[ \t]*//' -e 's/[ \t]*$//' | grep python3 | xargs sudo apt-get --reinstall install

  • thunderbird: disable annoyances thunderbird for enterprise

  • upgrade all pip packages
    add --local for virtualenvs

    pip install -U `pip list --outdated | awk '{ print $1}'`
    
  • use tar to back up a server
    crude, but effective

    :::bash
    tar -zcvpf /archive/full-backup-`date '+%d-%B-%Y'`.tar.gz --directory / \
    --exclude=mnt --exclude=proc --exclude=var/spool/squid --exclude=sys \
    --exclude=cdrom --exclude=sys --exclude=tmp
    

    or from serverpronto.com

    :::bash
    tar cvpzf /backups/backup.tgz --exclude=/proc --exclude=/lost+found \
    --exclude=/backups --exclude=/dev --exclude=/sys --exclude=/boot/grub \
    --exclude=/etc/fstab --exclude=/etc/network/interfaces \
    --exclude=/etc/udev/rules.d/70-persistent-net.rules /
    

    or

    :::bash
    tar cvpzf /root/backups.tgz --exclude=/root --exclude=/proc \
    --exclude=/lost+found --exclude=/backups --exclude=/dev \
    --exclude=/sys --exclude=/boot/grub --exclude=/etc/fstab \
    --exclude=/etc/network/interfaces
    

rsync

  • basic rsync -avs --delete -h . user@domain.tld:/home/user/server
  • copy directory structure only rsync -av -f"+ */" -f"- *" /path/to/src /path/to/dest/
  • ignore version control files(warning: ignores folders named core) rsync --cvs-exclude

git

  • set up new repo

    :::bash
    git init
    git add .
    git commit -m "commit message"
    git remote add remotename user@domain.tld:/home/user/proj.git
    git push --set-upstream remotename master
    # on server
    git --bare init
    
  • update submodules
    git pull --recurse-submodules
    with proper commits: git submodule update --recursive

  • change author info
    from github:
    git clone --bare https://github.com/user/repo.git
    cd repo.git
    run git-author-rewrite.sh, check logs
    git push --force --tags origin 'refs/heads/*'

Media

  • convert ogg to mp3
    for name in *.ogg; do ffmpeg -i "$name" -ab 128k -map_metadata 0:s:0 "${name/.ogg/.mp3}"; done;

  • convert flac to mp3(fish shell)

    for i in (ls *.flac)
      ffmpeg -i $i -ab 320k -map_metadata 0 -id3v2_version 3 $i.flac
    end
    
  • reduce video size
    ffmpeg -i input.mp4 -preset medium -c:v libx265 -crf 28 -c:a copy output.mp4

  • change volume ffmpeg -i input.mp3 -af "volume=0.8" output.mp3

  • save all mp3s from an m3u file
    wget -r 'url.m3u' -O - | wget -nd -N -i - && ls *.mp3 *.ogg *.flac *.wav > play.m3u

  • split audio files without reencoding
    install mp3splt
    mp3splt -S 5 input.mp3 (-D directory): split into 5 equal parts
    mp3splt -t 10.30 input.mp3: split into files of minutes.seconds

  • ffmpeg for video file size reduction
    ffmpeg -i input.mp4 -c:a copy -vcodec libx264 -crf 20 output.mp4

  • remove audio tracks from an mkv file
    mkvmerge --identify input.mkv (get track info)
    mkvmerge -o output.mkv --audio-tracks 2 input.mkv
    other usage: mkvtoolnix documentation

  • merge video and subtitles into container file mkvmerge -o output.mkv --default-track 0 --language 0:eng subtitles.srt input.mp4

  • audio players
    from ubuntuusers
    place a .is_audio_player file in root directory of audio player

  • splitting flac files (from ubuntugenius)
    install cuetools, shntool and flac packages
    run cuebreakpoints album.cue | shnsplit -o flac album.flac and then(after converting to mp3) cuetag album.cue *.mp3
    note: enclose filesnames containing special characters with double quotes

Typography

  • create specific unicode characters CTRL-SHIFT-U <NUMBER> ENTER
  • via compose key Set Right control(makes most sense) as compose key using gnome-tweak-tool, then either look up the corresponding sequence here or try to guess it, the mappings make sense most of the time
    (No need to keep pressing the compose modifier while entering)
    Examples:
    • . . for (ellipsis)
    • ~ n for ñ(eñe)
    • ? ? for ¿(inverted ?)
    • C C C P(uppercase!) for (hammer & sickle)

Display management

  • start display manager (xmonad, …)
    in .xinitrc

    :::bash
    xrandr -s 0
    xmonad &
    wait
    
  • set brightness xrandr --output DVI0 --brightness 0.5
    also use arandr

  • turn off display xset dpms force off, combine with lock: xset dpms force off; slock

System deployment

  • manipulate installed packages
    sudo dpkg --get-selections > packages.txt to back up
    then sudo dpkg --clear-selections to set all to deinstall
    sudo dpkg --set-selections < packages.txt to import the saved package list
    sudo aptitude install or sudo apt-get -u dselect-upgrade(much faster)

fish shell

  • ‘clear cache’ and speed up: remove ~/.config/fish/fishd*
  • save alias: alias la 'ls -la' and then funcsave <func>
  • clipboard: kill with CTRL-w or ALT-BCKSP -> kill ring.
    Paste:CTRL-y, cycle: ALT-y
  • man dmenu has a good explanation of movement
  • use ALT and left/right to move through directory history
  • use z with tab completion for a fuzzy ncurses-like file manager

Shortcuts from emacs-mode

Shortcut Action
TAB and SHIFT+TAB completion cycling
CTRL+a and CTRL+e jump to beginning/end of line
forward: ALT+f
backward: ALT+b
move by words
left: CTRL+b
right: CTRL+f
moving, autosuggestions
up: CTRL+p
down: CTRL+n
searching command history
(ALT with up/down only for tokens)
CTRL+h
CTRL+d
CTRL+j
backspace
forward delete
enter
CTRL+c
CTRL+k
CTRL+u
deletes whole line
cursor to end
beginning to cursor
CTRL+w
ALT+d
deletes the next part
deletes last part
ALT+l list contents of current dir
ALT+p pipe output to less for paged output
ALT+w show command description
CTRL+t
ALT+t
switches (transposes) the last two characters
the last two words

Fix specific application problems or configure for desired mode of operation

  • apt-get without diffs diff files are great for low bandwidth connections but they are not needed most of the time
    either run apt-get update with -o Acquire::Pdiffs=false
    or edit /etc/apt/apt.conf to include Acquire::PDiffs "false";
  • apt: disable automatic updates edit /etc/apt/apt.conf.d/10periodic and set APT::Periodic::Update-Package-Lists "0";
  • fix alsa problems (sudo) alsactl init
  • pandoc generate proper html with utf-8 encoding pandoc -f markdown infile.mkd -t html -s -o outfile.html
  • pandoc: website to epub pandoc -s -r html http://url.tld/page -o epub
  • try to explicitly log into an ssh server using password instead of an ssh key ssh -o PreferredAuthentications=keyboard-interactive -o \ PubkeyAuthentication=no user@host
  • gnome, systemd dependency madness on ubuntu from shallowsky.com
    install equivs package
    run equivs-control, edit Package to package name, Version to 2:42 or something high, Maintainer to some name and <email@address.com>
    run equivs-build and install with dpkg -i <packagename>
    Use this to remove e.g. evolution-data-server
  • package that a file belongs to dpkg -S file
  • disable bootup of services on ubuntu update-rc.d cups remove (note: at least for now, systemd might change this)
  • let apt ignore dependencies edit /var/lib/dpkg/status and remove Depends: entries (note: only short-term, this will get overwritten)
  • flask install directly from github tarball to use flask * commands
  • use sudo without entering your password all the time sudo visudo
    user ALL=NOPASSWD: ALL
    however, this is very insecure, so it is better to just whitelist specific commands:
    user ALL=(root) NOPASSWD: /usr/bin/command [args]
    %group ALL=(root) NOPASSWD: /path/to/comand [args]
  • sudo password asterisks Defaults env_reset,pwfeedback in visudo
  • pip problems cryptography need libssl-dev on ubuntu
    cffi needs python-cffi
    Pillow needs libjpeg-dev
  • stop mongodb from allocating 3GB of space (shutdown mongodb), edit /etc/mongodb.conf and insert smallfiles=true
    remove /var/lib/mongodb/journal/\*, (startup mongodb)
  • partition woes
    • partprobe /dev/sdX or kpartx /dev/sdX
    • try hdparm -Z /dev/sdX or sfdisk -R /dev/sdX
  • QT5
    • set ENVVAR with export QT_AUTO_SCREEN_SCALE_FACTOR=0.8 in /etc/profile.d/qt.sh or try with .bash_profile, .profile, .bashrc etc.
    • install libqt5libqgtk2 qt5-image-formats-plugins qtwayland5 qt5-qmltooling-plugins
    • run qt5ct

Power management and laptop handling

  • power saving
    • install laptop-mode-tools or, preferably tlp
    • install powertop and set toggles
    • utilize your desktop environment’s built-in solution, or run pm-suspend, pm-hibernate or pm-suspend-hybrid like an animal
    • edit /etc/systemd/logind.conf and set HandleLidSwitch=ignore, then run systemctl restart systemd-logind.service clean up boot with update-rc.d or disable systemd units, use the Startup applications program to manage autostarts

Fix problems pertaining to specific hardware configurations

  • wifi on Asus EEE PC 1000H

    install firmware-ralink

    edit /etc/network/interfaces

    auto wlan0
    iface wlan0 inet dhcp
        wpa-ssid YOUR-SSID-HERE
        wpa-psk YOUR-PASSWORD-HERE
    /etc/init.d/networking restart
    iwlist
    iwconfig -a
    iwconfig wlan0 down
    ifconfig wlan0 hw ether 01:23:45:67:89:00
    ifconfig wlan0 up
    

    I don’t use my old netbook any more, I hear OpenBSD has good drivers for wireless networking. Go have fun installing that one.

  • mitigate problems with intel ac 7260

    edit /etc/modprobe.d/iwlwifi-11n.conf and add options iwlwifi 11n_disable=8 (or disable=1)
    (at runtime) run sudo rmmod {iwlmvm,iwlwifi}, then sudo modprobe iwlwifi 11n_disable=1, sudo modprobe iwlmvm
    edit /etc/default/crda and change REGDOMAIN= to your country code
    note: this still does not fix all problems, especially eduroam is still wonky

  • bluetooth
    purge gnome-bluetooth
    install bluez-tools bluetooth blueman bluez-hcidump python-bluez and pulseaudio-module-blueooth (for audio)
    run sudo hciconfig hci0 reset and sudo pactl load-module module-blueotooth-discover
    if applicable, edit /etc/bluetooth/main.conf