NoSQL – Gangnam Style
It's so silly i had to blog it 🙂
... Read more
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
# http://torum.net/2010/01/kyotocabinet-alpha-release/ ulimit -u unlimited #zlib compression sudo apt-get install -y zlib1g-dev build-essential #lzma compression sudo apt-get install -y lzma liblzma-dev #lzo compression sudo apt-get install -y liblzo2-dev sudo apt-get remove -y libtokyocabinet-dbg libtokyocabinet-dev tokyocabinet-bin libtokyocabinet9 sudo apt-get install -y xz-utils #lua #sudo apt-get install -y build-essential lua5.2 liblua5.2-0 liblua5.2-0-dbg liblua5.2-dev sudo apt-get remove -y lua5.2 liblua5.2-0 liblua5.2-0-dbg liblua5.2-dev sudo apt-get remove -y lua50 liblua50 liblua50-dbg liblua50-dev sudo apt-get install -y lua5.1 liblua5.1 liblua5.1-dev # that was the fix sudo apt-get install -y zlib1g zlib1g-dev #/*** #apt-get install zlib-devel -y --force-yes # checking for zlib.h... no # configure: error: zlib.h is required #****/ #kyoto cabinet build mkdir -p /tmp/build cd /tmp/build version="1.2.76" # download wget http://fallabs.com/kyotocabinet/pkg/kyotocabinet-$version.tar.gz # extract tar xvzf kyotocabinet-$version.tar.gz # build and install cd kyotocabinet-$version ./configure --enable-zlib --enable-lzo --enable-lzma make sudo make install |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
#kyototycoon, works with lua 5.1 not 5.2 or later! mkdir -p /tmp/build cd /tmp/build version="0.9.56" wget http://fallabs.com/kyototycoon/pkg/kyototycoon-$version.tar.gz tar xvzf kyototycoon-$version.tar.gz cd kyototycoon-$version #./configure --with-kc --enable-lua #error fix #ktdbext.h:274:29: error: ‘getpid’ was not declared in this scope # append line #include <unistd.h> #http://code.google.com/p/modpagespeed/issues/detail?id=420 if [ "`grep 'include <unistd.h>' ./ktdbext.h`" == "" ] ; then sed -i -e 's/#define _KTDBEXT_H/#define _KTDBEXT_H\n#include <unistd.h>\n/g' ./ktdbext.h fi; #error fix #myscript.cc:3435:33: error: ‘lua_objlen’ was not declared in this scope #"Ipe requires Lua 5.1, and does not currently work with Lua 5.2." #http://sourceforge.net/apps/mantisbt/ipe7/view.php?id=113 #include "myconf.h" # build and install sudo ./configure --with-kc=/usr/local/kyotocabinet --enable-lua --with-lua=/usr/lib/x86_64-linux-gnu/ make sudo make install |
1 2 |
sudo add-apt-repository -y ppa:e2defrag/ppa sudo add-apt update |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# set version and base directory version="4.4.0" basedir=~/build/solr # create directory for Solr server binaries mkdir $basedir cd $basedir # download solr binaries > 70 MB wget http://apache.mirror.clusters.cc/lucene/solr/$version/solr-$version.zip # extract zip file unzip -oq solr-$version.zip # move versioned directory mv solr-$version solr |
1 2 3 4 5 6 |
# set base directory basedir=~/build/solr cd $basedir/solr/example/ # start solr server - uses port localhost:8983 java -jar start.jar |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
# set lucene version version="4.4.0" # set base directories basedir=~/build/lucene/lucene-$version indexdir=$basedir/data/index # create direct for Lucene code and binaries mkdir -p $indexdir cd $basedir/.. # download lucene if [ ! -e lucene-$version.zip ]; then wget http://apache.imsam.info/lucene/java/$version/lucene-$version.zip fi; # extract unzip -oq lucene-$version.zip # create directory to save the search index mkdir -p $basedir/data/index # start demo indexing cd lucene-$version # set classpath classpath=$classpath:$basedir/demo/lucene-demo-$version.jar classpath=$classpath:$basedir/core/lucene-core-$version.jar classpath=$classpath:$basedir/analysis/common/lucene-analyzers-common-$version.jar # Do index the directory with lucene documentation java -classpath $classpath org.apache.lucene.demo.IndexFiles -index $indexdir -docs $basedir/docs # check content of index directory echo $indexdir ls -lh $indexdir |
1 |
sudo apt-get install -y eclipse |
Fixes many missing library errors
1 2 3 4 |
if [ "`grep '/usr/local/lib' /etc/ld.so.conf`" == "" ] ; then sudo sh -c ' printf "\n/usr/local/lib" >> /etc/ld.so.conf'; fi; sudo ldconfig |
1 |
egrep "^14:59" access.log|grep exectime| sed 's/.*14:59.\(..\).*/\1/g'|sort |uniq -c|sort -n |
1 |
egrep "^14:" access.log|grep exectime| sed 's/.*14:\(.....\).*/\1/g'|sort |uniq -c|sort -n |
1 |
grep exectime access.log | awk '{print $1,$16,$7}' > se1.log |
1 |
tail -f /var/logs/access.log|grep exectime | awk '{print $1,$16,$7}' |
1 |
grep "job processed" res.log | awk '{ print $1" "$2}'|sort -n|uniq -c|less |
1 2 3 4 5 6 7 8 9 |
6 2013-01-31 18:39:44 4 2013-01-31 18:39:45 34 2013-01-31 18:39:46 150 2013-01-31 18:39:47 11 2013-01-31 18:39:48 20 2013-01-31 18:39:49 1 2013-01-31 18:39:50 17 2013-01-31 18:39:51 4 2013-01-31 18:39:52 |
1 |
for z in *.zip; do unzip -f $z; done |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
cat genre.txt ./014/735/54/00000000000001473554.xml <Genre name="Pop"> ./014/735/10/00000000000001473510.xml <Genre name="Pop"> ./014/726/18/00000000000001472618.xml <Genre name="Dance"> ./014/726/26/00000000000001472626.xml <Genre name="Rock"> ./014/726/08/00000000000001472608.xml <Genre name="Rap"> ./014/726/42/00000000000001472642.xml <Genre name="90's Rock"> |
1 |
grep Genre genre.txt |sed 's/.*Genre name="\(.*\).>.*/\1/g'|sort|uniq –c |
1 2 |
grep Genre genre.txt |sed 's/.*Genre name="\(.*\).>.*/\1/g'\ |sort|uniq -c|awk '{ print $2"\t"$1 }'|sort |sed 's/&apos//g' |
1 |
grep Genre genre.txt |sed 's/.*Genre name="\(.*\).>.*/\1/g’ |
1 2 |
#file extension for f in *; do fn="${f%.*}";echo "fn=$fn,f=$f"; done; |
1 2 3 4 |
tr '[A-Z]' '[a-z]' # to lower case sed 's/]]/]/g' # replace ]] with ] awk '{print tolower($0)}' # change whole string to lower sed 's/^[0-9]*//g' # replace all numbers with nothing |
1 2 3 4 |
for i in 187 135; do echo "--server${i}--"; scp -p -r /dev/mydata myuser@myhost${i}.net:/dev ; done |
1 |
echo 'psql -a -d mydatabase -p 5432 -c "\d mytable;"'| ssh myuser@myserverhost.net 'sudo su - postgres -c "bash -x -- " ' |
1 |
echo 'psql -a -d mydatabase -p 5432 -c select id from mytable where id = 393050;"'| ssh myuser@myserverhost.net 'sudo su - postgres -c "bash -x -- " ' |
1 2 3 4 5 6 |
lsof -i lsof -i | less # firefox example lsof -i | grep firefox |
1 2 3 4 |
while [ "1" ] ; do clear; lsof -i; sleep 1; done; # firefox example while [ "1" ] ; do clear; lsof -i | grep firefox; sleep 1; done; |
1 2 3 4 5 6 7 |
netstat |grep localhost while [ "1" ] ; do clear; netstat |grep localhost; sleep 1; done; |
1 |
less +F $(date "+/var/logs/vmware_%F.log") |
1 2 3 4 5 6 |
du -xh DIRPATH |egrep "^[0-9\.]+[GT]\s" # Examples du -xh ~/ |egrep "^[0-9\.]+[GT]\s" sudo du -xh / |egrep "^[0-9\.]+[GT]\s" |
1 |
top -c |
1 |
sudo apt-get install -y sysstat |
1 |
iostat -k 3 |
1 |
top -c -b |
1 |
SELECT * FROM pg_stat_activity order by client_addr; |
1 |
ps -fA|grep postgres |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 |
PS(1) User Commands PS(1) NAME ps - report a snapshot of the current processes. SYNOPSIS ps [options] DESCRIPTION ps displays information about a selection of the active processes. If you want a repetitive update of the selection and the displayed information, use top(1) instead. This version of ps accepts several kinds of options: 1 UNIX options, which may be grouped and must be preceded by a dash. 2 BSD options, which may be grouped and must not be used with a dash. 3 GNU long options, which are preceded by two dashes. Options of different types may be freely mixed, but conflicts can appear. There are some synonymous options, which are functionally identical, due to the many standards and ps implementations that this ps is compatible with. Note that "ps -aux" is distinct from "ps aux". The POSIX and UNIX standards require that "ps -aux" print all processes owned by a user named "x", as well as printing all processes that would be selected by the -a option. If the user named "x" does not exist, this ps may interpret the command as "ps aux" instead and print a warning. This behavior is intended to aid in transitioning old scripts and habits. It is fragile, subject to change, and thus should not be relied upon. By default, ps selects all processes with the same effective user ID (euid=EUID) as the current user and associated with the same terminal as the invoker. It displays the process ID (pid=PID), the terminal associated with the process (tname=TTY), the cumulated CPU time in [DD-]hh:mm:ss format (time=TIME), and the executable name (ucmd=CMD). Output is unsorted by default. The use of BSD-style options will add process state (stat=STAT) to the default display and show the command args (args=COMMAND) instead of the executable name. You can override this with the PS_FORMAT environment variable. The use of BSD-style options will also change the process selection to include processes on other terminals (TTYs) that are owned by you; alternately, this may be described as setting the selection to be the set of all processes filtered to exclude processes owned by other users or not on a terminal. These effects are not considered when options are described as being "identical" below, so -M will be considered identical to Z and so on. Except as described below, process selection options are additive. The default selection is discarded, and then the selected processes are added to the set of processes to be displayed. A process will thus be shown if it meets any of the given selection criteria. EXAMPLES To see every process on the system using standard syntax: ps -e ps -ef ps -eF ps -ely To see every process on the system using BSD syntax: ps ax ps axu To print a process tree: ps -ejH ps axjf To get info about threads: ps -eLf ps axms To get security info: ps -eo euser,ruser,suser,fuser,f,comm,label ps axZ ps -eM To see every process running as root (real & effective ID) in user format: ps -U root -u root u To see every process with a user-defined format: ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm ps -Ao pid,tt,user,fname,tmout,f,wchan Print only the process IDs of syslogd: ps -C syslogd -o pid= Print only the name of PID 42: ps -p 42 -o comm= SIMPLE PROCESS SELECTION a Lift the BSD-style "only yourself" restriction, which is imposed upon the set of all processes when some BSD-style (without "-") options are used or when the ps personality setting is BSD-like. The set of processes selected in this manner is in addition to the set of processes selected by other means. An alternate description is that this option causes ps to list all processes with a terminal (tty), or to list all processes when used together with the x option. -A Select all processes. Identical to -e. -a Select all processes except both session leaders (see getsid(2)) and processes not associated with a terminal. -d Select all processes except session leaders. --deselect Select all processes except those that fulfill the specified conditions (negates the selection). Identical to -N. -e Select all processes. Identical to -A. g Really all, even session leaders. This flag is obsolete and may be discontinued in a future release. It is normally implied by the a flag, and is only useful when operating in the sunos4 personality. -N Select all processes except those that fulfill the specified conditions (negates the selection). Identical to --deselect. T Select all processes associated with this terminal. Identical to the t option without any argument. r Restrict the selection to only running processes. x Lift the BSD-style "must have a tty" restriction, which is imposed upon the set of all processes when some BSD-style (without "-") options are used or when the ps personality setting is BSD-like. The set of processes selected in this manner is in addition to the set of processes selected by other means. An alternate description is that this option causes ps to list all processes owned by you (same EUID as ps), or to list all processes when used together with the a option. PROCESS SELECTION BY LIST These options accept a single argument in the form of a blank-separated or comma-separated list. They can be used multiple times. For example: ps -p "1 2" -p 3,4 -123 Identical to --pid 123. 123 Identical to --pid 123. -C cmdlist Select by command name. This selects the processes whose executable name is given in cmdlist. -G grplist Select by real group ID (RGID) or name. This selects the processes whose real group name or ID is in the grplist list. The real group ID identifies the group of the user who created the process, see getgid(2). -g grplist Select by session OR by effective group name. Selection by session is specified by many standards, but selection by effective group is the logical behavior that several other operating systems use. This ps will select by session when the list is completely numeric (as sessionsare). Group ID numbers will work only when some group names are also specified. See the -s and --group options. --Group grplist Select by real group ID (RGID) or name. Identical to -G. --group grplist Select by effective group ID (EGID) or name. This selects the processes whose effective group name or ID is in grouplist. The effective group ID describes the group whose file access permissions are used by the process (see getegid(2)). The -g option is often an alternative to --group. p pidlist Select by process ID. Identical to -p and --pid. -p pidlist Select by PID. This selects the processes whose process ID numbers appear in pidlist. Identical to p and --pid. --pid pidlist Select by process ID. Identical to -p and p. --ppid pidlist Select by parent process ID. This selects the processes with a parent process ID in pidlist. That is, it selects processes that are children of those listed in pidlist. -s sesslist Select by session ID. This selects the processes with a session ID specified in sesslist. --sid sesslist Select by session ID. Identical to -s. t ttylist Select by tty. Nearly identical to -t and --tty, but can also be used with an empty ttylist to indicate the terminal associated with ps. Using the T option is considered cleaner than using t with an empty ttylist. -t ttylist Select by tty. This selects the processes associated with the terminals given in ttylist. Terminals (ttys, or screens for text output) can be specified in several forms: /dev/ttyS1, ttyS1, S1. A plain "-" may be used to select processes not attached to any terminal. --tty ttylist Select by terminal. Identical to -t and t. U userlist Select by effective user ID (EUID) or name. This selects the processes whose effective user name or ID is in userlist. The effective user ID describes the user whose file access permissions are used by the process (see geteuid(2)). Identical to -u and --user. -U userlist Select by real user ID (RUID) or name. It selects the processes whose real user name or ID is in the userlist list. The real user ID identifies the user who created the process, see getuid(2). -u userlist Select by effective user ID (EUID) or name. This selects the processes whose effective user name or ID is in userlist. The effective user ID describes the user whose file access permissions are used by the process (see geteuid(2)). Identical to U and --user. --User userlist Select by real user ID (RUID) or name. Identical to -U. --user userlist Select by effective user ID (EUID) or name. Identical to -u and U. OUTPUT FORMAT CONTROL ... see man ps |