Gugulethu Ncube « Gugulethu Ncube

About Gugulethu Ncube

IT enthusiast getting things done.

It's so silly i had to blog it 🙂 ... Read more

Lux IO is a small performant key value storage. It is B+-tree based and is quite ... Read more

Install Kyoto Cabinet

# http://torum.net/2010/01/kyotocabinet-alpha-release/

ulimit -u unlimited

#zlib compression

sudo apt-get install -y zlib1g-dev build-essential

#lzma compression

sudo apt-get install -y lzma liblzma-dev

#lzo compression

sudo apt-get install -y liblzo2-dev

sudo apt-get remove -y libtokyocabinet-dbg libtokyocabinet-dev tokyocabinet-bin libtokyocabinet9

sudo apt-get install -y xz-utils

#lua

#sudo apt-get install -y build-essential lua5.2 liblua5.2-0 liblua5.2-0-dbg liblua5.2-dev

sudo apt-get remove -y lua5.2 liblua5.2-0 liblua5.2-0-dbg liblua5.2-dev

sudo apt-get remove -y lua50 liblua50 liblua50-dbg liblua50-dev

sudo apt-get install -y lua5.1 liblua5.1 liblua5.1-dev

# that was the fix

sudo apt-get install -y zlib1g zlib1g-dev

#/***

#apt-get install zlib-devel -y --force-yes

# checking for zlib.h... no

# configure: error: zlib.h is required

#****/

#kyoto cabinet build

mkdir -p /tmp/build

cd /tmp/build

version="1.2.76"

# download

wget http://fallabs.com/kyotocabinet/pkg/kyotocabinet-$version.tar.gz

# extract

tar xvzf kyotocabinet-$version.tar.gz

# build and install

cd kyotocabinet-$version

./configure --enable-zlib --enable-lzo --enable-lzma

make

sudo make install

Language bindings available Lua: ... Read more

Prepare Install Kyoto Cabinet Installation

#kyototycoon, works with lua 5.1 not 5.2 or later!

mkdir -p /tmp/build

cd /tmp/build

version="0.9.56"

wget http://fallabs.com/kyototycoon/pkg/kyototycoon-$version.tar.gz

tar xvzf kyototycoon-$version.tar.gz

cd kyototycoon-$version

#./configure --with-kc --enable-lua

#error fix

#ktdbext.h:274:29: error: ‘getpid’ was not declared in this scope

# append line #include <unistd.h>

#http://code.google.com/p/modpagespeed/issues/detail?id=420

if [ "`grep 'include <unistd.h>' ./ktdbext.h`" == "" ] ; then

sed -i -e 's/#define _KTDBEXT_H/#define _KTDBEXT_H\n#include <unistd.h>\n/g' ./ktdbext.h

fi;

#error fix

#myscript.cc:3435:33: error: ‘lua_objlen’ was not declared in this scope

#"Ipe requires Lua 5.1, and does not currently work with Lua 5.2."

#http://sourceforge.net/apps/mantisbt/ipe7/view.php?id=113

#include "myconf.h"

# build and install

sudo ./configure --with-kc=/usr/local/kyotocabinet --enable-lua --with-lua=/usr/lib/x86_64-linux-gnu/

make

sudo make install

Troubleshooting Problem: ... Read more

Add a PPA Example: ppa:e2defrag/ppa

1 2	sudo add-apt-repository -y ppa:e2defrag/ppa sudo add-apt update

Remove ... Read more

Download Server

# set version and base directory

version="4.4.0"

basedir=~/build/solr

# create directory for Solr server binaries

mkdir $basedir

cd $basedir

# download solr binaries > 70 MB

wget http://apache.mirror.clusters.cc/lucene/solr/$version/solr-$version.zip

# extract zip file

unzip -oq solr-$version.zip

# move versioned directory

mv solr-$version solr

Start Server

# set base directory

basedir=~/build/solr

cd $basedir/solr/example/

# start solr server - uses port localhost:8983

java -jar start.jar

Index ... Read more

Download Lucene and Index Files

# set lucene version

version="4.4.0"

# set base directories

basedir=~/build/lucene/lucene-$version

indexdir=$basedir/data/index

# create direct for Lucene code and binaries

mkdir -p $indexdir

cd $basedir/..

# download lucene

if [ ! -e lucene-$version.zip ]; then

wget http://apache.imsam.info/lucene/java/$version/lucene-$version.zip

fi;

# extract

unzip -oq lucene-$version.zip

# create directory to save the search index

mkdir -p $basedir/data/index

# start demo indexing

cd lucene-$version

# set classpath

classpath=$classpath:$basedir/demo/lucene-demo-$version.jar

classpath=$classpath:$basedir/core/lucene-core-$version.jar

classpath=$classpath:$basedir/analysis/common/lucene-analyzers-common-$version.jar

# Do index the directory with lucene documentation

java -classpath $classpath org.apache.lucene.demo.IndexFiles -index $indexdir -docs $basedir/docs

# check content of index directory

echo $indexdir

ls -lh $indexdir

Demo - File ... Read more

the start bash script is among others: /etc/bash.bashrc The aliases file can be ... Read more

The easiest: #Application Menu -> Settings -> Main Menu Otherwise download and ... Read more

The easiest way to install Eclipse in Ubuntu is through the standard repository

1	sudo apt-get install -y eclipse

Unfortunately, ... Read more

Fixes many missing library errors

if [ "`grep '/usr/local/lib' /etc/ld.so.conf`" == "" ] ; then

sudo sh -c ' printf "\n/usr/local/lib" >> /etc/ld.so.conf';

fi;

sudo ldconfig

About Gugulethu Ncube

IT enthusiast getting things done.

Method 1 - rm alias to mv to Trash # in your aliases - e.g. ~/.bash_aliases - add ... Read more

1	egrep "^14:59" access.log\|grep exectime\| sed 's/.14:59.$..$./\1/g'\|sort \|uniq -c\|sort -n

1	egrep "^14:" access.log\|grep exectime\| sed 's/.14:$.....$./\1/g'\|sort \|uniq -c\|sort -n

1	grep exectime access.log \| awk '{print $1,$16,$7}' > se1.log

1	tail -f /var/logs/access.log\|grep exectime \| awk '{print $1,$16,$7}'

1	grep "job processed" res.log \| awk '{ print $1" "$2}'\|sort -n\|uniq -c\|less

Result:

6 2013-01-31 18:39:44

4 2013-01-31 18:39:45

34 2013-01-31 18:39:46

150 2013-01-31 18:39:47

11 2013-01-31 18:39:48

20 2013-01-31 18:39:49

1 2013-01-31 18:39:50

17 2013-01-31 18:39:51

4 2013-01-31 18:39:52

The Virtualbox Menu bar disappears if the virtual machine is running in "scale mode". Resizing ... Read more

Extract multiple zip files

1	for z in *.zip; do unzip -f $z; done

Extract multiple ... Read more

Simple Conversion of FLV to MP4 using FFMPEG ffmpeg chews the input buffers - ... Read more

Source File

cat genre.txt

./014/735/54/00000000000001473554.xml

./014/735/10/00000000000001473510.xml

./014/726/18/00000000000001472618.xml

./014/726/26/00000000000001472626.xml

./014/726/08/00000000000001472608.xml

./014/726/42/00000000000001472642.xml

1	grep Genre genre.txt \|sed 's/.Genre name="$.$.>.*/\1/g'\|sort\|uniq –c

1 2	grep Genre genre.txt \|sed 's/.Genre name="$.$.>.*/\1/g'\ \|sort\|uniq -c\|awk '{ print $2"\t"$1 }'\|sort \|sed 's/&apos//g'

1	grep Genre genre.txt \|sed 's/.Genre name="$.$.>.*/\1/g’

Reload

1	pg_ctl -D . reload

Stop - soft

1	pg_ctl -D . stop -m i

Stop ... Read more

1 2	#file extension for f in ; do fn="${f%.}";echo "fn=$fn,f=$f"; done;

About Gugulethu Ncube

IT enthusiast getting things done.

tr '[A-Z]' '[a-z]' # to lower case

sed 's/]]/]/g' # replace ]] with ]

awk '{print tolower($0)}' # change whole string to lower

sed 's/^[0-9]*//g' # replace all numbers with nothing

About Gugulethu Ncube

IT enthusiast getting things done.

Copy Data to multiple machines

for i in 187 135; do

echo "--server${i}--";

scp -p -r /dev/mydata myuser@myhost${i}.net:/dev ;

done

Execute Scripts ... Read more

1	echo 'psql -a -d mydatabase -p 5432 -c "\d mytable;"'\| ssh myuser@myserverhost.net 'sudo su - postgres -c "bash -x -- " '

1	echo 'psql -a -d mydatabase -p 5432 -c select id from mytable where id = 393050;"'\| ssh myuser@myserverhost.net 'sudo su - postgres -c "bash -x -- " '

About Gugulethu Ncube

IT enthusiast getting things done.

lsof -i

lsof -i | less

# firefox example

lsof -i | grep firefox

check every second

while [ "1" ] ; do clear; lsof -i; sleep 1; done;

# firefox example

while [ "1" ] ; do clear; lsof -i | grep firefox; sleep 1; done;

netstat |grep localhost

while [ "1" ] ; do

clear;

netstat |grep localhost;

sleep 1;

done;

1	less +F $(date "+/var/logs/vmware_%F.log")

About Gugulethu Ncube

IT enthusiast getting things done.

du - file space usage GB sized files and directories

du -xh DIRPATH |egrep "^[0-9\.]+[GT]\s"

# Examples

du -xh ~/ |egrep "^[0-9\.]+[GT]\s"

sudo du -xh / |egrep "^[0-9\.]+[GT]\s"

Size ... Read more

1. Top

top -c

top -c

column wa => io wait 2. IOStat Installation

sudo apt-get install -y sysstat

1	sudo apt-get install -y sysstat

iostat -k 3

1	iostat -k 3

IO ... Read more

Top Every Second -Version 1

top -c -b

top -c -b

Top Every Second ... Read more

Define an Alias The first example causes ls -l to be executed when the command ll ... Read more

Postgres Database – Who is Currently Connected

1	SELECT * FROM pg_stat_activity order by client_addr;

1	ps -fA\|grep postgres

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

PS(1) User Commands PS(1)

NAME

ps - report a snapshot of the current processes.

SYNOPSIS

ps [options]

DESCRIPTION

ps displays information about a selection of the active processes. If you want a repetitive update of the selection

and the displayed information, use top(1) instead.

This version of ps accepts several kinds of options:

1 UNIX options, which may be grouped and must be preceded by a dash.

2 BSD options, which may be grouped and must not be used with a dash.

3 GNU long options, which are preceded by two dashes.

Options of different types may be freely mixed, but conflicts can appear. There are some synonymous options, which

are functionally identical, due to the many standards and ps implementations that this ps is compatible with.

Note that "ps -aux" is distinct from "ps aux". The POSIX and UNIX standards require that "ps -aux" print all

processes owned by a user named "x", as well as printing all processes that would be selected by the -a option. If

the user named "x" does not exist, this ps may interpret the command as "ps aux" instead and print a warning. This

behavior is intended to aid in transitioning old scripts and habits. It is fragile, subject to change, and thus

should not be relied upon.

By default, ps selects all processes with the same effective user ID (euid=EUID) as the current user and associated

with the same terminal as the invoker. It displays the process ID (pid=PID), the terminal associated with the

process (tname=TTY), the cumulated CPU time in [DD-]hh:mm:ss format (time=TIME), and the executable name (ucmd=CMD).

Output is unsorted by default.

The use of BSD-style options will add process state (stat=STAT) to the default display and show the command args

(args=COMMAND) instead of the executable name. You can override this with the PS_FORMAT environment variable. The

use of BSD-style options will also change the process selection to include processes on other terminals (TTYs) that

are owned by you; alternately, this may be described as setting the selection to be the set of all processes filtered

to exclude processes owned by other users or not on a terminal. These effects are not considered when options are

described as being "identical" below, so -M will be considered identical to Z and so on.

Except as described below, process selection options are additive. The default selection is discarded, and then the

selected processes are added to the set of processes to be displayed. A process will thus be shown if it meets any

of the given selection criteria.

EXAMPLES

To see every process on the system using standard syntax:

ps -e

ps -ef

ps -eF

ps -ely

To see every process on the system using BSD syntax:

ps ax

ps axu

To print a process tree:

ps -ejH

ps axjf

To get info about threads:

ps -eLf

ps axms

To get security info:

ps -eo euser,ruser,suser,fuser,f,comm,label

ps axZ

ps -eM

To see every process running as root (real & effective ID) in user format:

ps -U root -u root u

To see every process with a user-defined format:

ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm

ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm

ps -Ao pid,tt,user,fname,tmout,f,wchan

Print only the process IDs of syslogd:

ps -C syslogd -o pid=

Print only the name of PID 42:

ps -p 42 -o comm=

SIMPLE PROCESS SELECTION

a Lift the BSD-style "only yourself" restriction, which is imposed upon the set of all processes when some

BSD-style (without "-") options are used or when the ps personality setting is BSD-like. The set of processes

selected in this manner is in addition to the set of processes selected by other means. An alternate

description is that this option causes ps to list all processes with a terminal (tty), or to list all

processes when used together with the x option.

-A Select all processes. Identical to -e.

-a Select all processes except both session leaders (see getsid(2)) and processes not associated with a terminal.

-d Select all processes except session leaders.

--deselect

Select all processes except those that fulfill the specified conditions (negates the selection). Identical to

-N.

-e Select all processes. Identical to -A.

g Really all, even session leaders. This flag is obsolete and may be discontinued in a future release. It is

normally implied by the a flag, and is only useful when operating in the sunos4 personality.

-N Select all processes except those that fulfill the specified conditions (negates the selection). Identical to

--deselect.

T Select all processes associated with this terminal. Identical to the t option without any argument.

r Restrict the selection to only running processes.

x Lift the BSD-style "must have a tty" restriction, which is imposed upon the set of all processes when some

BSD-style (without "-") options are used or when the ps personality setting is BSD-like. The set of processes

selected in this manner is in addition to the set of processes selected by other means. An alternate

description is that this option causes ps to list all processes owned by you (same EUID as ps), or to list all

processes when used together with the a option.

PROCESS SELECTION BY LIST

These options accept a single argument in the form of a blank-separated or comma-separated list. They can be used

multiple times. For example: ps -p "1 2" -p 3,4

-123 Identical to --pid 123.

123 Identical to --pid 123.

-C cmdlist

Select by command name. This selects the processes whose executable name is given in cmdlist.

-G grplist

Select by real group ID (RGID) or name. This selects the processes whose real group name or ID is in the

grplist list. The real group ID identifies the group of the user who created the process, see getgid(2).

-g grplist

Select by session OR by effective group name. Selection by session is specified by many standards, but

selection by effective group is the logical behavior that several other operating systems use. This ps will

select by session when the list is completely numeric (as sessionsare). Group ID numbers will work only when

some group names are also specified. See the -s and --group options.

--Group grplist

Select by real group ID (RGID) or name. Identical to -G.

--group grplist

Select by effective group ID (EGID) or name. This selects the processes whose effective group name or ID is

in grouplist. The effective group ID describes the group whose file access permissions are used by the

process (see getegid(2)). The -g option is often an alternative to --group.

p pidlist

Select by process ID. Identical to -p and --pid.

-p pidlist

Select by PID. This selects the processes whose process ID numbers appear in pidlist. Identical to p and

--pid.

--pid pidlist

Select by process ID. Identical to -p and p.

--ppid pidlist

Select by parent process ID. This selects the processes with a parent process ID in pidlist. That is, it

selects processes that are children of those listed in pidlist.

-s sesslist

Select by session ID. This selects the processes with a session ID specified in sesslist.

--sid sesslist

Select by session ID. Identical to -s.

t ttylist

Select by tty. Nearly identical to -t and --tty, but can also be used with an empty ttylist to indicate the

terminal associated with ps. Using the T option is considered cleaner than using t with an empty ttylist.

-t ttylist

Select by tty. This selects the processes associated with the terminals given in ttylist. Terminals (ttys,

or screens for text output) can be specified in several forms: /dev/ttyS1, ttyS1, S1. A plain "-" may be used

to select processes not attached to any terminal.

--tty ttylist

Select by terminal. Identical to -t and t.

U userlist

Select by effective user ID (EUID) or name. This selects the processes whose effective user name or ID is in

userlist. The effective user ID describes the user whose file access permissions are used by the process (see

geteuid(2)). Identical to -u and --user.

-U userlist

Select by real user ID (RUID) or name. It selects the processes whose real user name or ID is in the userlist

list. The real user ID identifies the user who created the process, see getuid(2).

-u userlist

Select by effective user ID (EUID) or name. This selects the processes whose effective user name or ID is in

userlist.

The effective user ID describes the user whose file access permissions are used by the process (see

geteuid(2)). Identical to U and --user.

--User userlist

Select by real user ID (RUID) or name. Identical to -U.

--user userlist

Select by effective user ID (EUID) or name. Identical to -u and U.

OUTPUT FORMAT CONTROL

...

see man ps

About Gugulethu Ncube

IT enthusiast getting things done.

Gugulethu Ncube

me, myself and code

Author Archives: Gugulethu Ncube

About Gugulethu Ncube

NoSQL – Gangnam Style

LuxIO – Installation and Test of the NoSQL key value database

Kyoto Cabinet – Installation with Lua, compression using zip, lzo and lzma

Kyoto Tycoon – Installation with Lua and Memcached Protocol

Ubuntu – Add and Delete a PPA

Apache Solr 4 – Getting Started

Lucene 4 – Getting Started – Demo File Search

Linux – Aliases for root user

Linux – XFCE Menu editor and lxmed

Linux – Install Eclipse IDE

Linux – Adding /usr/local/lib to /etc/ld.so.conf

About Gugulethu Ncube

Ubuntu – rm – Delete to Trash

Unix – Count Requests Per Seconds From Access Log File

Virtualbox – How To Fix The Missing Menu bar on Ubuntu and Windows

Unix – Extract multiple rar and zip files

ffmpeg – Convert Adobe Flash FLV Files to MP4 Files

Unix – grep, sort , uniq

Postgres – Tips and Tricks

Unix – Extract File Extension and Filename from Filepath

About Gugulethu Ncube

Unix – tolower in bash, awk and sed

About Gugulethu Ncube

Unix – Seqential Remote Calls to Several Machines

Postgres – Executing SQL-Statement in several remote machines sequentially

About Gugulethu Ncube

Unix – Check and Monitor Open Ports and established Connections

Unix – Less

About Gugulethu Ncube

Unix – disk and file size monitoring with du, df and stat

Linux – IO Wait Monitoring – Top, IOStat and WMStat

Unix – Executing Top once or Every X Seconds

Unix – Show, Create and Delete Command Alias

Postgres – Which users are Currently Connected to the database

Unix – ps – finding running processes

About Gugulethu Ncube