Tips & Tricks (Linux/Vim/Git/Programming)

In this post you can find various simple tricks and tips for Linux, Vim, Git, Python and many other stuff. All the material provided in this page has been suggested by many different sources such as Command Line Magic, Mastering Vim, kernel_perspective, nixCraft, and Linux Today. The material is free and can be redistributed and/or modified. There is no any warranty that they work for you or are suitable to your need. The author of this page is not responsible for any damage this material may cause.

How to count the lines of specific files in a dir (Linux)⌗

$ find . -name "*.c" | sed 's/.*/"&"/' | xargs wc -l    # assumed C files

Override the Tmux session’s numbering manually (Tmux)⌗

$ tmux new -s 1 -d -P

How to check if file A contains all lines of file B (Linux)⌗

$ comm -13 <(sort -u fileA) <(sort -u fileB)
# if the output is empty that means all the lines of fileB are in the fileA to0

How to delete all files except one or few files with extensions (Linux)⌗

$ rm -v !("filaneme_youd_like_to_keep")     # verbose mode

$ rm -i !("foo"|"bar")   # interactive mode - keep files foo and bar

$ rm -v (*.pdf | *.odt)  # keep files with extensions pdf and odt

How to avoid nested loops (Python)⌗

from itertools import product


x = [1, 2, 3, 4]
y = [6, 7]
z = [8, 9, 10]

for i, j, k in product(x, y, z):
    ... # do something with x, y, and z

Eval function (Python)⌗

def foo(x, y, operator):
    return eval(f'{x} {operator} {b}')

x, y = 3, 6
operator = "+"
print(foo(x, y, operator))
9

$ sudo apt install samba        # Install samba packages on Ubuntu/Debian
$ mkdir samba_shared_folder     # Create a temporary folder
$ vim /etc/samba/smb.conf

Add the following section to the smb.conf file.

[samba_share]
    comment = Samba on local network
    path = /home/username/path_to_samba_shared_folder
    read only = no
    browsable = yes

Then run the following commands:

$ sudo service smbd restart
$ sudo ufw allow samba      # net filter firewall
$ sudo smbpasswd -a username

To access the SAMBA shared folder, either use any manager such as Nautilus or through the terminal:

$ smbclient -L //hostname/samba_shared_folder

$ mkdir temproot
$ cd temproot
$ ln -s FILE_TO_BE_SHARED
$ python3 -mhttp.server     # file now is accessible at http://other_machine_ip:8000/

Compress an executable using UPX (Linux)⌗

$ upx -n compressed_exec_name source_exec_name

Copy/move a file into the directory you were just in (Linux)⌗

$ cp (or mv) FILENAME "$OLDPWD/"

Find big files (Linux)⌗

$ find / -type f -size +XXXM    # where XXX is the desired size in MB

Detect any vulnerabilities on your system (Linux)⌗

$ grep -r . /sys/devices/system/cpu/vulnerabilities/

Remove duplicates from Python’s list (Python)⌗

collection = [1, 1, 1, 2, 3, 4, 4, 4, 4, 4, 7, 8]
collection = list(set(collection))

print(collection)
[1, 2, 3, 4, 7, 8]

Merge two dictionaries in Python (Python)⌗

dict1 = {'a': 1}
dict2 = {'b': 2}

dict = {**dict1, **dict2}

Filter a list in Python (Python)⌗

mylist = [1, 2, 3, 4, 5, 6, 7, 8]

filtered_list = list(filter(lambda x: x%2==0, mylist))

print(filtered_list)
[1, 3, 5, 7]

Sort a Python dictionary by values⌗

grades = {'Bob':3.5, 'Alice': 4.0, 'Foo': 2.2}
sorted_grades = {k:v for k, v in sorted(grades.items(), key=lambda x:x[1])}

print(sorted_grades)
{'Foo': 2.2, 'Bob': 3.5, 'Alice': 4.0}

Sort a Python dictionary by keys⌗


grades = {'Bob':3.5, 'Alice': 4.0, 'Foo': 2.2}
sorted_grades = {key:grades[key] for key in sorted(grades.keys())}

print(sorted_grades)
{'Alice': 4.0', Bob': 3.5, 'Foo': 2.2}

Map, Filter, and Reduce (Python)⌗

# MAP
def square(x):
    return x*x

x = [2, 3, 4, 5, 6]
res = map(square, x)
# res = map(lambda y:y*y, x)

print(res)
[4, 9, 16, 25, 36]

# FILTER
def less_than_10(x):
    if x < 10:
        return x

x = [2, 3, 4, 5, 10, 12, 14]
res = list(filter(less_than_10, x))

print(res)
[2, 3, 4, 5]

# REDUCE

def addition(x,y):
    return x + y
        
from functools import reduce

x = [1, 2, 3, 4, 5]
res = reduce(addition, x))

print(res)
15

Python string methods (Python)⌗


string = "Hello, World!"

print(string.isalpha())    # check if all characters are letters
False

print(string.replace("Hello", "Hi"))    # replace Hello with Hi
Hi, World!

print(string.count("World"))    # count the occurrences of word World
1 

print(string.find("!"))     # return the position of character '!'
12

print(string.upper())       # capitalize all characters
HELLO, WORLD!

print(string.lower())       # convert all characters to lower-case
hello, world!

$ systemctl reboot --firmware-setup

Joining dictinary keys and values to a list in Python (Programming)⌗

'_'.join("{!s}_{!r}".format(key, val) for (key, val) in d.items())

Check the health status of your hard drive (Linux)⌗

$ badblocks -w -s -o error.log /dev/sdX     # get X using lsblk

How to load (reload) modules (iPython)⌗

%load_ext autoreload
%autoreload 2

Power-off an external hard drive (Linux)⌗

$ udiskctl power-off -b /dev/sdX    # obtain X using lsblk

rsync your data (Linux)⌗

$ rsync -avzhP src dst --exclude-from=file.txt --include=".git/config"

Recover data (Linux)⌗

$ ddrescue -dr3 /dev/sdX imagename.image logfile

Random execution of a Linux command (Linux)⌗

$ [ $((RANDOM % 2)) -eq 0 ] && command

Open a file in VIM with no compatibility (Vim)⌗

$ vim -u NONE -u NORC fname

IO devices latency (Linux)⌗

$ ioping /tmp/
$ ioping -c 12 /tmp/    # runs for 12 rounds
$ sudo ioping -R /dev/sdX         # seek rate
$ sudo ioping -RL /dev/sdX        # sequential speed

Copy a file to the clipboard (Linux)⌗

$ xclip -sel clip < file

Import date in VIM (Vim)⌗

:read !date

Import the output of a Linux command in Vim (Vim)⌗

:let @a=system("command")
In insert mode press Ctrl+R and then a

Convert a bunch of png files to a video (Linux)⌗

$ ffmpeg -framerate 1/5 -patter_type glob -i "*.png" -vf "fps=95,format=yuv420p" output.mp4

Convert a photo into ASCII art (Linux)⌗

$ jp2a photo.jpg | tee photo.ascii

Read .gz files without extracting them (Linux)⌗

$ zmore file.gz
$ zless file.gz

Get memory information (Linux)⌗

$ sudo lshw -C memory -short

$ sudo dmidecode -t memory

How to load huge files in Python (Programming)⌗

import mmap
f = open("HUGE.txt")
mm = mmap.mmap(f.filend, length=0, access=mmap.ACCESS_READ, offset=0)
print(mm[-1000:])

Git credential storage (Git)⌗

$ git config --global credential.helper cache '--timeout S'  # S -> seconds

Include a help flag in Makefiles (Programming)⌗

.PHONY:
    help

help:
    cat makefile | grep -oP "^#\K(.*)"

Copy a file into multiple directories (Linux)⌗

echo dir1 dir2 | xargs -n 1 cp file
parallel cp file ::: dir1 dir2

How to run a ELF (Programming)⌗

$ touch file
$ cp /bin/ls .
$ ./ls
file ls
$ chmod -x ./ls
$ ./ls
-bash: ./ls: Permission denied
$ strings ./ls | head -n 1
/lib64/ld-linux-x86-64.so.2
$ /lib64/ld-linux-x86-64.so.2 ./ls
file ls

Set an alert message when a job finishes (Linux)⌗

$ { execute_something_that_takes_time; xmessage DONE; }&

Dennis Ritchie (Linux/Programmin)⌗

$ curl -L git.io/unix

Play tetris (Linux)⌗

$ ssh netris\.rocketnine\.space

Python scope (Programming)⌗

LEGB:

Local (within a function)
Enclosing functions locals
Global (module)
Built-in (python) preassigned names

C Pointers (Programming)⌗

ptr++ Evaluates ptr and then increments ptr by 1xbase_type_size
*ptr++ Evaluates ptr, increments it, and dereferences the evaluated value
int *ptr[10] Array of 10 pointers to integer (int *)
int (*ptr)[10] Pointer to an array of 10 integers

Delete the last column of a text file (Linux)⌗

$ awk 'NF {NF-=1};1'< input_file > output_file
$ awk 'NF {NF--};1'< input_file > output_file

Automatic variables in Make (Programming)⌗

$@: The target filename.
$*: The target filename without the file extension.
$<: The first prerequisite filename.
$^: The filenames of all the prerequisites, separated by spaces, discard duplicates.
$+: Similar to $^, but includes duplicates.
$?: The names of all prerequisites that are newer than the target, separated by spaces.