User Tools

Site Tools


art:command_cheat_sheet

Command Cheat Sheet

Sha1 Hash A Password From Linux Console

echo -n "typethepasswordhere" | sha1sum | awk '{print $1}'

GCC & GDB

gcc -g -pthread -o output_file source_file.c header_file.h -l ssl -l crypto
gdb core output_file
bt

Core dumps are disabled on some configurations. GCC arguments depend on libraries used in source code

Return 100 Biggest Files

sudo du -ah . | sort -n -r | head -n 100
sudo find / -type f -printf '%s %p\n' 2>/dev/null | sort -nr | head -100

Remove Duplicates From ~/.bash_history

awk '!seen[$0]++' ~/.bash_history | sort -u > ~/.bash_history.new
mv ~/.bash_history.new ~/.bash_history

Rector

Install composer…

From the current directory of the script to test…

composer require rector/rector --dev
vendor/bin/rector init
// define sets of rules
$rectorConfig->sets([
  LevelSetList::UP_TO_PHP_82
]);
./vendor/bin/rector process ./ --dry-run

or

./vendor/bin/rector process ./

TV and Radio Server

The TV and Radio Ganino promotion for easier success uses Darwin Streaming Server (DSS) by Apple. Darwin Streaming Server (DSS), was the first open sourced RTP/RTSP streaming server. It was released March 16, 1999 and is a fully featured RTSP/RTP media streaming server capable of streaming a variety of media types including H.264/MPEG-4 AVC, MPEG-4 Part 2 and 3GP.

How to install the Darwin media streaming server on Linux (Ubuntu)

Rachael Bond Saturday, 26 June 2010 Linux

The instructions have now been updated for Ubuntu 12.04 LTS. Homepage for DSS is http://dss.macosforge.org/ also tested on Linux Mint 14

Ensure that you having the following dependencies installed:

~$ sudo apt-get install gcc g++ make patch

You can download the latest version of the Darwin server package from Apple.

~$ wget http://dss.macosforge.org/downloads/DarwinStreamingSrvr6.0.3-Source.tar
or:
~$ wget http://www.ganino.com/files/DarwinStreamingSrvr6.0.3-Source.tar

To additional patches are required to use Darwin on Ubuntu and a custom install script.

Once again, if you prefer to download straight to your server use either:

~$ wget http://www.ganino.com/files/dss-6.0.3.patch
~$ wget http://www.ganino.com/files/dss-hh-20080728-1.patch

Before installing Darwin create its user and group (hyphen may to be change from – to -):

~$ sudo addgroup -system qtss
~$ sudo adduser -system -no-create-home -ingroup qtss qtss

Next unpack the .tar file:

~$ tar -xvf DarwinStreamingSrvr6.0.3-Source.tar

and apply the two patches:

~$ patch -p0 < dss-6.0.3.patch
~$ patch -p0 < dss-hh-20080728-1.patch

Move into the Darwin directory:

~$ cd DarwinStreamingSrvr6.0.3-Source

and delete the Install file:

~$ rm Install

Download the new Install file:

~$ wget http://www.ganino.com/files/Install

and change its permissions:

~$ chmod +x Install

For Ubuntu 12.04 it would appear that there is a compiling problem. This can be resolved by editing line 8 in Makefile.POSIX (in DarwinStreamingSrvr6.0.3-Source) and changing it from:

LIBS = $(CORE_LINK_LIBS) -lCommonUtilitiesLib -lQTFileLib

to:

LIBS = $(CORE_LINK_LIBS) -lCommonUtilitiesLib -lQTFileLib -ldl

After this Darwin can be built:

~$ ./Buildit

and install:

~$ sudo ./Install

To allow Darwin to run automatically on boot, download either:

~$ wget http://www.ganino.com/files/darwin-streaming-server

and change its permissions:

~$ chmod +x darwin-streaming-server

Move this file into /etc/init.d:

~$ sudo mv darwin-streaming-server /etc/init.d/darwin-streaming-server

and then update rc.d to autostart it:

~$ sudo update-rc.d darwin-streaming-server defaults

Darwin requires the ports TCP: 554, 1220 and UDP: 6970-6999 open in order to work. If you are using Shorewall edit /etc/shorewall/rules and add these lines:

ACCEPT net fw tcp 554 ACCEPT net fw tcp 1220 ACCEPT net fw udp 6970:6999

Finally direct your browser to http://yourserver:1220 to check that Darwin is working and to enter your admin passwords. If your server is going to be exclusively for RTSP streaming then select port 80 for use by Darwin – otherwise leave blank as this will interfere with Apache.

You are now ready to upload .mp4 and .mov files for streaming…. however, make sure that your .mp4 files have been correctly encoded and hinted.

Broadcasting a professional image ensures success and easier acceptance. It is accessible from any device that has internet anywhere and at anytime globally. Keep broadcasting.

Encoding for DSS - Re-encoding & Hinting

Encoding is key, encoding videos to specifications that server expects. For example encoding video in h264 and audio in aac. You will need to re-encode all files so that they are identical in every way. I use Wondershare Video Converter Ultimate.

The profile is Mp4:

Video Codec: H.263 Audio Codec: AA2 Video Size: Bit Rate 512Kbps (means every user that downloads the stream at 512kps will get smooth video, the server must also be able to stream @ 512kbps to every user.

After re-encoding you must hint the video. The file will not play until you hint the file.

~$ apt-get install gpac

will provide the MP4Box command

~$ MP4Box -hint thefiletohint.mp4

a check on the video will now show the rtp streams section that a non-hinted video does not have.

~$ ffmpeg -i file-we-require-info-on.ext

To extract Audio From A Video Using ffmpeg

~$ ffmpeg -i “whatever.format” -vn -ac 2 -ar 44100 -ab 320k -f mp3 output.mp3

starting the server point your browser to

http://www.ganino.com:1220

to access the stream use standard rtp port

rtsp://www.ganino.com/tv.sdp

to access the mp3 stream

http://www.ganino.com:8000/radio

SELinux and Firewall will not let it through. Disable SELinux for this test by editing /etc/SElinux/config and setting it to disabled. /etc/init.d/iptables stop to stop the firewall, later adding the exceptions rather than disabling.

QuickTime Streaming Server 4 and Darwin Streaming Server 4 use the following TCP ports:

  • 80: HTTP
  • 554: RTSP
  • 6970 -9999: used for dynamic (announced) UDP broadcasts
  • 7070: RTSP
  • 8000, 8001: MP3 streaming
  • 10,000-20,000: buffer space
  • 20,000-65,535: static SDP default range (user defined)

the comments relating to chkconfig are important as chkconfig will read them regardless of the hash usually meaning comments are ok to disregard.

$ chkconfig --add myscript 
$ chkconfig --level 2345 myscript on 
$ chkconfig --list | grep myscript 

Editing Playlist With A Command

for f in /usr/local/movies//*.mp3; do echo "\"$f\" 5"; done >> radio.playlist

file being /var/streaming/playlists/radio/radio.playlist

Batch Rename Remove Char

for i in *\]* ; do mv -v "$i" "${i/\]/}" ; done

Create HTML page from directory

for f in *.mp3; do echo "<a href=\"/music/beethoven/$f\">$f</a><br />"; done >> page.txt

Copy Recursive Into One Directory

find . -iname '*.mp3' -type f -exec cp {} target_dir/ \;

PF Rules

pfctl -f /etc/pf.conf

Rename folders to numbers

#!/bin/sh
COUNTER=0;
for i in `find . -type d`; do
COUNTER=$((COUNTER + 1))
mv ${i} $COUNTER;
done

Remove spaces from directories

find -name "* *" -type f | rename 's/ /_/g

Shell command to bulk change file extensions in a directory (Linux)

1. Change from one extension to another The command below will rename all files with the extension .php4 to .php

for f in *.php4; do mv $f `basename $f .php4`.php; done;
  

2. Add (append) an extension to all files The command below add the extension .txt to all files in the directory

for f in *; do mv $f `basename $f `.txt; done;
  

2. Remove (delete) an extension from all files The command below remove the extension .txt from all files in the directory

for f in *.txt; do mv $f `basename $f .txt`; done;

Artists Scripts

#! /bin/bash

for entry in * 
do 
echo "{{:artists:art_via_denmark:$entry?direct&100|}}" 
done 

Pinger

#!/bin/bash
for a in {0..255}
 do
  for b in {0..255}
   do
    for c in {0..255}
     do
      for d in {0..255}
       do
        ping -c 4 -t 10 $a.$b.$c.$d >> $a.$b.$c.$d
       #echo $a.$b.$c.$d
      done
     done
    done
   done

or

#!/bin/bash
for a in {0..255}
 do
  for b in {0..255}
   do
    for c in {0..255}
     do
      for d in {0..255}
       do
        echo "====== $a.$b.$c.$d ======" >> $a.$b.$c.$d
        echo "===== Ping =====" >> $a.$b.$c.$d
        ping -c 4 $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Nmap =====" >> $a.$b.$c.$d
        nmap -T4 -A -v -p 1-65535
        echo "===== Associated IP =====" >> $a.$b.$c.$d
        dig  $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Associated Websites =====" >> $a.$b.$c.$d
        dig -x $a.$b.$c.$d >> $a.$b.$c.$d
        echo "===== Activity =====" >> $a.$b.$c.$d
        echo "Edit page" >> $a.$b.$c.$d
       done
      done
     done
    done

Dokuwiki Permissions

   
 {DOCUMENT_ROOT}/internet/conf/ is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/pages is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/attic is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media_attic is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/media_meta is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/meta is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/cache is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/locks is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/index is not writable by DokuWiki.
 {DOCUMENT_ROOT}/internet/data/tmp is not writable by DokuWiki.

Dokuwiki Enwiki

 cat enwiki-latest-pages-articles.xml | ./mwimport | mysql -f -D database -u user -ppassword
 bzcat dewiki-20120603-pages-articles.xml.bz2 | perl mwimport.pl | mysql -f -u [USERNAME] -p [DATABASE]
 

Replace Rename Folder Removing Whitespace

for f in *; do mv "$f" `echo $f | tr ' ' '_'`; done
for i in `find . -type d`; do new_name=`echo $i | tr '[A-Z]' '[a-z]'`; mv $i $new_name ; done
for i in `find . -type f`; do new_name=`echo $i | tr '[A-Z]' '[a-z]'`; mv $i $new_name ; done

Duplicate Concat

:g/^/norm yyp

Yet another one(shorter):

:%s/.*/&\r&

Another one:

:%!sed p

/etc/my.cnf bind-address = 127.0.0.1 skip-external-locking This also prevents MySQL from listening on the external network interfaces; note: the skip-external-locking directive should be used instead of the deprecated skip-locking.

As a first step after basic configuration, we then need to install the default databases, change the password of the MySQL root user (don't take my passwords as an example!) and answer a few questions:

# /usr/local/bin/mysql_install_db [ … ] # /usr/local/bin/mysqld_safe & [ … ] # /usr/local/bin/mysql_secure_installation [ … ] Enter current password for root (enter for none): <Enter> OK, successfully used password, moving on… [ … ] Set root password? [Y/n] Y New password: root Re-enter new password: root Password updated successfully! [ … ] Remove anonymous users? [Y/n] Y … Success! [ … ] Disallow root login remotely? [Y/n] Y … Success! [ … ] Remove test database and access to it? [Y/n] Y - Dropping test database… … Success! - Removing privileges on test database… … Success! [ … ] Reload privilege tables now? [Y/n] Y … Success! [ … ] #

pf.conf ruleset

#### First declare a couple of variables ####
### Outgoing tcp / udp port ####
### 43 - whois, 22 - ssh ###
tcp_services = "{ ssh, smtp, domain, www, https, 22, ntp, 43,ftp, ftp-data}"
udp_services = "{ domain, ntp }"
### allow ping / pong ####
icmp_types = "{ echoreq, unreach }"
#### define tables. add all subnets and ips to block
table <blockedip> persist file "/etc/pf.block.ip.conf"
martians = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24, 0.0.0.0/8, 240.0.0.0/4 }"
### admin server ranges ###
adminrange = "112.220.11.0/23"
# connected to internet
ext_if = "em1"  
# connected to vpn / lan
int_if = "em0"
##### ftp proxy
#proxy="127.0.0.1"
#proxyport="8021"
#### Normalization
#scrub provides a measure of protection against certain kinds of attacks based on incorrect handling of   packet fragments
scrub in all
#### NAT and RDR start
#nat-anchor "ftp-proxy/*"
#rdr-anchor "ftp-proxy/*"
# redirect ftp traffic
#rdr pass proto tcp from any to any port ftp -> $proxy port $proxyport
# Drop incoming everything 
block in all 
block return 
# keep stats of outgoing connections
pass out keep state
# We need to have an anchor for ftp-proxy
#anchor "ftp-proxy/*"
# unlimited traffic  for loopback and lan / vpn
set skip on {lo0, $int_if}
# activate spoofing protection for all interfaces
block in quick from urpf-failed
#antispoof is a common special case of filtering and blocking. This mechanism protects against activity from spoofed or forged IP addresses
antispoof log for $ext_if
#Block RFC 1918 addresses
block drop in log (all)  quick on $ext_if from $martians to any
block drop out log (all) quick on $ext_if from any to $martians
# Block all ips
# pfctl -t blockedip -T show
block drop in log (all)  quick on $ext_if from <blockedip> to any
block drop out log (all) quick on $ext_if from any to <blockedip>
# allow outgoing 
pass out on $ext_if proto tcp to any port $tcp_services
pass out on $ext_if proto udp to any port $udp_services
# Allow trace route
pass out on $ext_if inet proto udp from any to any port 33433 >< 33626 keep state
# Allow admin to get into box
pass in on $int_if from $adminrange to any
# Allow incoming ssh, http, bind traffic
# pass in  on $ext_if proto tcp from any to any port 25
pass in on $ext_if proto tcp from any to any port ssh  flags S/SA synproxy state
pass in on $ext_if proto udp from any to any port domain 
pass in on $ext_if proto tcp from any to any port domain flags S/SA synproxy state
pass in on $ext_if proto tcp from any to any port http flags S/SA synproxy modulate state
pass inet proto icmp all icmp-type $icmp_types keep state
## add your rule below ##

AVI to MP4

avconv -i A\ Blade\ in\ the\ Dark.avi -c:v libx264 -c:a copy A\ Blade\ in\ the\ Dark.mp4

Find Files Larger Than A Certain Size

find ./ -xdev -type f -size +500M

Some Useful Linux Unix Commands

Mono In The Background

nohup mono myexe.exe &

SSH Command In The Background Then Exit

tmux
CTRL+B D
tmux attach   #to come back to the command in the next session

Join Multiple MP4 Using GUI

mkvtoolnix-gui  #apt-get install ...

Video Convertor

sudo add-apt-repository ppa:ffmulticonverter/stable
sudo apt-get update
sudo apt-get install ffmulticonverter
 

Generate Thumbnails From Images

mogrify -resize 200 *.png

Ebay Search All (old)

(th*,po*,a,and,at,condition,has,in,is,on,of,off,or,for,from,new,old,on,paypal,pick,up,to,was,where,when,will,with,%,$,1,2,3,4,5,6,7,8,9,0)

CCTV recorder

Turn a webcam into a CCTV recorder

streamer -q -c /dev/video0 -f rgb24 -r 24 -t 02:30:00 -o outfile.avi &

BSD Nameserver Bind Reload

rndc reload

Restart SSHD

kill -HUP `cat /var/run/sshd.pid`

Chmod Directories

find ./ -type d -exec chmod 755 {} \;

Chmod Files

find ./ -type f -exec chmod 644 {} \;

Remove ^M

:%s/^V^M//g

Make Thumbnails

mogrify -resize 80x80 -background white -gravity center -extent 80x80 -format png -quality 75 *.png

Images From PDF

convert -density 900 -depth 8 -quality 100 Voynich_Manuscript.pdf Voynich_Manuscript.png

Convert Directory to MP3

find -name "*.flac" -exec avconv -i {} -acodec libmp3lame -ab 320k {}.mp3 \;

Merge PDF

pdftk *.pdf cat output 1234567.pdf;

Sample Daemon Startup Script On Red Hat Systems

#!/bin/bash
# chkconfig: 2345 20 80
# description: Description comes here....

# Source function library.
. /etc/init.d/functions

start() {
  # code to start app comes here 
}

stop() {
  # code to stop app comes here 
}

case "$1" in 
 start)
     start
     ;;
 stop)
     stop
     ;;
 retart)
     stop
     start
     ;;
 *)
      echo "Usage: $0 {start|stop|restart}"
esac

exit 0

Create Site Mirror

wget -mk -w 20 http://www.example.com/

SCP Various Commands

Copy the file “foobar.txt” from a remote host to the local host

scp your_username@remotehost.edu:foobar.txt /some/local/directory

Copy the file “foobar.txt” from the local host to a remote host

scp foobar.txt your_username@remotehost.edu:/some/remote/directory

Copy the directory “foo” from the local host to a remote host's directory “bar”

scp -r foo your_username@remotehost.edu:/some/remote/directory/bar

Copy the file “foobar.txt” from remote host “rh1.edu” to remote host “rh2.edu”

scp your_username@rh1.edu:/some/remote/directory/foobar.txt your_username@rh2.edu:/some/remote/directory/

Bzcat & enWiki

bzcat dewiki-20120603-pages-articles.xml.bz2 | perl mwimport.pl | mysql -f -u [USERNAME] -p [DATABASE]
cat enwiki-<date>.xml | perl mwimport.pl | mysql -f -u<admin name> -p<admin password> --default-character-set=utf8 <database name>

Mount BSD Drive From Linux

sudo mount -t ufs -r -o ufstype=44bsd /dev/sdb1 /mnt/

Copy multiple files and append to end of filename

for f in /etc/*.conf; do cp -v -- "$f" "$f.orig"; done

How to file split at a line number

wc -l 100 file

Create Sitemap (Dirty)

find ./ > sitemap.test

Open in gvim search and replace

./ 

with

<url><loc>http://www.yoursite.com/

Open in vim and run command

 :%s/$/\**##**/g \\

Open in gvim search and replace

  • *##**

with

</loc><lastmod>2014-03-05T22:44:12+11:00</lastmod></url>

Add the open and closing tags

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
 ~~~
 </urlset>
 

Vim Search and replace

 %s/<\/loc>/  \r<\/loc>/g
 

Split With Custom Prefix

 split -l 49999 --additional-suffix=.xml sitemap.test sitemap

Gzip Multiple

 gzip sitemap*.xml
 

Add Line TO start and end of mutiple file

 
 for file in sitemap*.xml; do
  sed -i '1i <?xml version="1.0" encoding="UTF-8"?>\r\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">' "$file" &&
  echo '</urlset>' >> "$file"
 done
 

Freqtrade

https://www.freqtrade.io/en/stable/

cd freqtrade
source ./.env/bin/activate

# Step 1 - Initialize user folder

freqtrade create-userdir --userdir user_data

# Step 2 - Create a new configuration file

freqtrade new-config --config config.json
freqtrade trade --config config.json --strategy SampleStrategy

# FreqUI is now built into Freqtrade, https://www.freqtrade.io/en/stable/rest-api/

yarn serve
art/command_cheat_sheet.txt · Last modified: 2023/01/31 08:32 by admin