CentOs - Tips and tricks

This post will hold the tips and tricks cheat sheet for things related to CentOs. I will add things to this post as and when I face issues. Please comment here if you would like to add sth to the list or if sth is wrong. Thanks in advance.

I'm using CentOS release 6.3 (Final) version as a base for my VM. I usually get a basic OVA, deploy it to some ESXi host and add my stuff on top of it. That usually means I deploy my web applications, databases, maven repository, etc.

• When deployed to some host and powered on, no IP address is configured?

         In this version, network service is not started at boot time. Since my OVA is preconfigured my eth0 config looks like this:

DEVICE="eth0"
BOOTPROTO="dhcp"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"

    Look at this link for using other options like static IP or to set up DNS servers: http://linhost.info/2011/12/centos-6-has-no-network-connectivity/

    After configuring eth0, just fire up the following command

service network restart
 
  This would contact dhcp server and get your IP. Then ifconfig should list the interfaces for you with updated IP.

1. This really isn't just related to CentOS but here's to general Linux tips.

To find folder/file recursively, "find $PWD -type d -name 'target"

Vim Tips and Tricks

VIM ON MAC:

In Macbook, create  a file called $HOME/.vimrc and add the following:

 :syntax on  - To include syntax highlighting in vi as default
:map :w\|!python %  - To save and run python script with just pressing 
:set number - To display line numbers

SCAN AND RUN:

 I had to do performance test on a REST server via SoapUi. This is what I did, in SOAPUI(3.5.1) I created Test suite->Test steps->properties(added the txnId to be unique for subsequent calls)->Groovy script to increment the txnId->Test case which has the sample request->LoadTestcase where I set the delay between requests and total # of requests for n number of threads.

In the server, I added code on server's entry point to print the difference of start and end timestamp. So after the Load Test case is done, I had to extract those timestamps, send it to a script which would calculate the avg time for each request.

The tricky part was how to extract the timestamp difference from the server log. Here's the vi commands to achieve it. In my logs, I had the timestamp in milliseconds like "PERFORMANCE:: 120". So I copy the server log to temp log. Open it via vim editor. In command prompt,

cp server.log temp.log - do not modify the original server log
grep "PERFORMANCE::" temp.log - verify you the test ran successfully
grep -c "PERFORMANCE::" temp.log - count the total number of occurrences
vi temp.log - open the temp log in vi editor
:v/PERFORMANCE:: /d - in esc mode, delete all lines except our performance test line
:%s/^.*PERFORMANCE::\([^"]*\).*$/\1 - in the leftover lines, delete the words and leave the numbers alone
wc -l temp.log - verify the count is still the same, note it down as total#Requests
vi testscript.sh - open file to write script to add the numbers left in temp.log


The file content should be:

sum="0"
for i in `cat $1`;
do
sum=$[$sum + $i];
done
echo "sum = "$sum

chmod +x testscript.sh - to enable execution of the script
./testscript.sh temp.log - to print the sum
then in calculator sum/total#request = avg time taken by the server for each request.

ToNote:
:g/.*/d - to delete all the lines in a file in Vi.

Default $PATH value for Mac OS X

When I messed up the PATH variable in my Mac, I found a life saver here.

I'm reposting here the default value of $PATH env variable, in case you screwed up your $PATH.

Just execute, export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

But ideally, remember to do this before you change your PATH variable.

1. In CL, echo $PATH

2. copy the result in file that is already open

3. change your $PATH.

4. If something is wrong, in CL execute "export PATH=savedvalue"

Subclipse in Eclipse [MAC]

To install subclipse in Eclipse ide:

Help->Install new software->Add->http://subclipse.tigris.org/update_1.6.x

If you are behind firewall you would get error like this

Unable to connect to repository http://subclipse.tigris.org/update_1.6.x/content.xml Unable to connect to repository http://subclipse.tigris.org/update_1.6.x/content.xml Connection refused

Then do the following,

1. In Eclipse (go to Preferences > General > Network connections).
2. Select "Active Provider" as "Manual".
3. Select HTTP and click edit.
4. Enter the host and port [proxy host and port get it from office admin]
5. Select "Reuries Authentication" and enter the username and password.

6. Repeat Step 1 to 5 for Https.

   
   

   Src: http://stackoverflow.com/questions/4598167/eclipse-updates-not-working

Try Catch Finally

In case of sudden doubts, here are few pointers from specifications:

A try statement with a finally block is executed by first executing the try block. Then there is a choice:

• If execution of the try block completes normally, then the finally block is executed, and then there is a choice:
• If the finally block completes normally, then the try statement completes normally.
• If the finally block completes abruptly for reason S, then the try statement completes abruptly for reason S.
• If execution of the try block completes abruptly because of a throw of a value V, then there is a choice:
• If the run-time type of V is assignable to the parameter of any catch clause of the try statement, then the first (leftmost) such catch clause is selected. The value V is assigned to the parameter of the selected catch clause, and the Block of that catch clause is executed. Then there is a choice:
• If the catch block completes normally, then the finally block is executed. Then there is a choice:
• If the finally block completes normally, then the try statement completes normally.
• If the finally block completes abruptly for any reason, then the try statement completes abruptly for the same reason.
• If the catch block completes abruptly for reason R, then the finally block is executed. Then there is a choice:
• If the finally block completes normally, then the try statement completes abruptly for reason R.
• If the finally block completes abruptly for reason S, then the try statement completes abruptly for reason S (and reason R is discarded).
• If the run-time type of V is not assignable to the parameter of any catch clause of the try statement, then the finally block is executed. Then there is a choice:
• If the finally block completes normally, then the try statement completes abruptly because of a throw of the value V.
• If the finally block completes abruptly for reason S, then the try statement completes abruptly for reason S (and the throw of value V is discarded and forgotten).
• If execution of the try block completes abruptly for any other reason R, then the finally block is executed. Then there is a choice:
• If the finally block completes normally, then the try statement completes abruptly for reason R.
• If the finally block completes abruptly for reason S, then the try statement completes abruptly for reason S (and reason R is discarded).

Src: http://java.sun.com/docs/books/jls/third_edition/html/statements.html#14.20.2

If execution of the try block completes abruptly because of a throw of a value V, then there is a choice:

• If the run-time type of V is assignable to the Parameter of any catch clause of the try statement, then the first (leftmost) such catch clause is selected. The value V is assigned to the parameter of the selected catch clause, and the Block of that catch clause is executed. If that block completes normally, then the try statement completes normally; if that block completes abruptly for any reason, then the try statement completes abruptly for the same reason.

Src: http://java.sun.com/docs/books/jls/second_edition/html/statements.doc.html#24134

SVN : tips and tricks

1. Checkout only a single file from repository without having to download the entire directory.

Sometimes we want to work on only a single file out of 1000's of files in a repository folder. You don't have to download them all. You can do this

svn co --depth empty

cd

svn up

Thanks to source here. 

2. When you access subversion from Eclipse you will be prompted for username and password. You can select an option to cache the password so you don't have to enter them every time you connect to repository. Suppose if you want to reset the cache and to be prompted for password again, you need to manually delete the file containing the cached password. In Mac OSX, its

~/.subversion/auth/.svn.simple

Now restart the IDE, you will be prompted for password again.

3. If you want to ignore certain files from ever "accidentally" checked in to SVN, do the following. I'm talking about those .settings, .classpath, .project, target, log server.out, depedency-reduced-pom.xml, etc.

1. Go to ~/.subversion/config file
2. Find "global-ignores" line. It would be commented by default.
3. Add this instead "global-ignores = *.classpath *.project *.settings target

The location of config file based on the system:

In Mac/Linux:

~/.subversion/config or /etc/subversion/config

In Window:

%appdata%\subversion\config

Mac: TcpMon

What: TcpMon is a monitor tool that comes with Apache Axis package and can be downloaded.

Why: If you have a WS client or service that you want to monitor, use TcpMon. Lets say you have written a client for axis2 service as stated here. You also may need to look at the SOAP messages that are sent in the wire.

How:

1. Download TcpMon binary distribution here
2. Execute tcpmon-1.0-bin/build/tcpmon.sh . This will launch the tcpMon console.
3. In 'Admin' tab, enter 'Listen Port#:' Any_port_Number. Set 'Act as: Proxy'. And click Add. New tab as 'Port Any_port_Number' will come up where you can see all traffic.
4. Now you need to set proxy for the web based traffic. In Mac, to System Preferences->Network->Advanced->Proxies->Web proxy. Set 127.0.0.1 and Any_port_Number. Note: you may need to revert back to original settings when you are done with debugging.
5. Run the client in the eclipse. Go to the new tab in tcpmon and you can see the incoming and outgoing SOAP messages for debugging.


Thanks to the source!

Axis2 Maven Web service client Example

Aim: To learn how to generate java client stubs from WSDL using wsdl2java script of axis2. Use the client stubs to consume a web service using SOAP over HTTP.

Requirements:
1. you need WSDL of the service you wish to consume using your generated client.
2. If you are going to test both service generation and consumption in local system, download axis2 binary distribution.
3. SoapUI if you need to test the remote Server and its operations with sample requests.


Steps:

I followed this source to set up the axis2's http server and also download SimpleService.aar file to deploy. Here is the simple steps to follow:

1. Download and extract axis2 binary distribution to /usr/local, at the time of writing[Apr 14,2011] we have axis2-1.5.4. Set /usr/local/axis2-1.5.4 as AXIS_HOME. You can find the scripts in AXIS_HOME/bin folder. You need to copy .aar file of your service into repository/services folder.

2. Start the axis2 server by executing AXIS_HOME/bin/axis2server.sh. In a browser if you access, http://localhost:8080 it would list all services deployed on the axis2server. You can get the WSDL of the service from its links.

SimpleService.wsdl:
==============

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:ns1="http://org.apache.axis2/xsd" xmlns:ns="http://ws.apache.org/axis2" xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" targetNamespace="http://ws.apache.org/axis2">
<wsdl:types>
<xs:schema attributeFormDefault="qualified" elementFormDefault="qualified" targetNamespace="http://ws.apache.org/axis2">
<xs:element name="echo">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="args0" nillable="true" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="echoResponse">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="return" nillable="true" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
</wsdl:types>
<wsdl:message name="echoRequest">
<wsdl:part name="parameters" element="ns:echo"/>
</wsdl:message>
<wsdl:message name="echoResponse">
<wsdl:part name="parameters" element="ns:echoResponse"/>
</wsdl:message>
<wsdl:portType name="SimpleServicePortType">
<wsdl:operation name="echo">
<wsdl:input message="ns:echoRequest" wsaw:Action="urn:echo"/>
<wsdl:output message="ns:echoResponse" wsaw:Action="urn:echoResponse"/>
</wsdl:operation>
</wsdl:portType>
<wsdl:binding name="SimpleServiceSoap11Binding" type="ns:SimpleServicePortType">
<soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
<wsdl:operation name="echo">
<soap:operation soapAction="urn:echo" style="document"/>
<wsdl:input>
<soap:body use="literal"/>
</wsdl:input>
<wsdl:output>
<soap:body use="literal"/>
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
<wsdl:binding name="SimpleServiceSoap12Binding" type="ns:SimpleServicePortType">
<soap12:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
<wsdl:operation name="echo">
<soap12:operation soapAction="urn:echo" style="document"/>
<wsdl:input>
<soap12:body use="literal"/>
</wsdl:input>
<wsdl:output>
<soap12:body use="literal"/>
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
<wsdl:binding name="SimpleServiceHttpBinding" type="ns:SimpleServicePortType">
<http:binding verb="POST"/>
<wsdl:operation name="echo">
<http:operation location="SimpleService/echo"/>
<wsdl:input>
<mime:content type="text/xml" part="echo"/>
</wsdl:input>
<wsdl:output>
<mime:content type="text/xml" part="echo"/>
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
<wsdl:service name="SimpleService">
<wsdl:port name="SimpleServiceHttpSoap11Endpoint" binding="ns:SimpleServiceSoap11Binding">
<soap:address location="http://localhost:8080/axis2/services/SimpleService.SimpleServiceHttpSoap11Endpoint/"/>
</wsdl:port>
<wsdl:port name="SimpleServiceHttpSoap12Endpoint" binding="ns:SimpleServiceSoap12Binding">
<soap12:address location="http://localhost:8080/axis2/services/SimpleService.SimpleServiceHttpSoap12Endpoint/"/>
</wsdl:port>
<wsdl:port name="SimpleServiceHttpEndpoint" binding="ns:SimpleServiceHttpBinding">
<http:address location="http://localhost:8080/axis2/services/SimpleService.SimpleServiceHttpEndpoint/"/>
</wsdl:port>
</wsdl:service>
</wsdl:definitions>


Things you need to know about any WSDL is
a. What are the possible 'operation' from the service. Their input and output element type and their arguments.
b. What is the address Location of the service.

3. I use Maven2Eclipse plugin to generate and execute my project. Create a maven project. Update its pom.xml with the following:

<project>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-wsdl2code-maven-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<goals>
<goal>wsdl2code</goal>
</goals>
</execution>
</executions>
<configuration>
<packageName>axis.test</packageName>
<wsdlFile>src/main/wsdl/SimpleService.wsdl</wsdlFile>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-transport-local</artifactId>
<version>1.5.4</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-transport-http</artifactId>
<version>1.5.4</version>
</dependency>
<dependency>
<groupId>wsdl4j</groupId>
<artifactId>wsdl4j</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.schema</groupId>
<artifactId>XmlSchema</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.neethi</groupId>
<artifactId>neethi</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.axiom</groupId>
<artifactId>axiom-api</artifactId>
<version>1.2.11</version>
</dependency>
<dependency>
<groupId>org.apache.ws.commons.axiom</groupId>
<artifactId>axiom-impl</artifactId>
<version>1.2.11</version>
</dependency>
...
</dependencies>
...
</project>

These are the dependencies you will need to execute your Client stub generated by wsdl2java. The plugin is used to generate the stubs. When you run Maven Install on the project it should generate SimpleServiceCallbackHandler.java and SimpleServiceStub.java in target/generated-sources/../axis/test folder. Move it to your project's src folder under needed package name.

4. Here is a sample Client.java that you need to write to consume the SimpleService using the stubs we just generated.

Client.java:
========
package axis.test;

import java.rmi.RemoteException;

public class Client {

/**
* @param args
* @throws RemoteException
*/
public static void main(String[] args) {

try {
SimpleServiceStub stub = new SimpleServiceStub();

SimpleServiceStub.Echo request = new SimpleServiceStub.Echo();

request.setArgs0("This is my message");

SimpleServiceStub.EchoResponse response = stub.echo(request);

System.out.println("Response from Simple service: "+response.get_return());

}
catch(RemoteException ex) {
System.out.println(ex);
ex.printStackTrace();
}
}
}

This is a very simple client code to start with. You need to create an instance of stub class which would be ServiceName+Stub.java. You need to create a request object. It will be a inner class of stub. You can check wsdl for operation name(that would request name). Set the arguments for the operation request and verify you are complying the format specified in WSDL. In this case its a string to echo back. Similarly create instance of response to hold the response of the operation. You actually consume the service when you call the operation via stub object. get_return() method always returns the value of response object.


5. Run Client as Java application and make sure service is up by checking via browser. You will get response in console. Viola.


Possible errors/exceptions:

1. When the service is down, the SOAP Fault would say Connection refused.
2. There are 3 data binding frameworks possible. Default is adb(Axis data binding framework). You can specify it in in maven plugin for wsdl2code.

References:
[1]. Hello world with Apache Axis2

Ubuntu: Install Tomcat

1. Download the latest version of apache-tomcat from here.
2. "cd /downloaded/directory" --> In terminal, go to the downloaded directory.
   
3. "sudo cp apache* /usr/local" --> Copy the downloaded file to /usr/local (can be changed, I prefer local dir).
4. "sudo chmod a+rwx apache*" --> change the access permissions.
5. "sudo tar zxvf apache*" --> unzip the tar.gz file.
6. "export JAVA_HOME=/usr/local/java/jdk1.6.0_17" --> set up JAVA_HOME env variable
7. "export PATH=$JAVA_HOME/bin:$PATH" --> add java bin to path
8. "export CATALINA_HOME=/usr/local/apache-tomcat-6.0.32" --> optional to set up these env variables.
9. "cd $CATALINA_HOME/bin" --> go to bin folder to access scripts.
10. "sudo sh startup.sh" --> to manually start Tomcat
11. "localhost:8080" --> Go to browser and access the page to see configuration page of Tomcat.
12. If step 11 returns error, "cd $CATALINA_HOME/logs" --> go to logs folder and view "catalina..log" file to see what went wrong. If everything went well, you should see "INFO: Starting Coyote HTTP/1.1 on http-8080". If you see "java.net.BindException: Address already in use", shutdown and restart Tomcat.
    
13. "sudo sh $CATALINA_HOME/bin/shutdown.sh" --> to shutdown Tomcat.

Src: http://www.puschitz.com/InstallingTomcat.html#InstallingTomcatSoftware


EXTRA:

Mac: Installing Tomcat

src: http://developer.apple.com/internet/java/tomcat1.html

1. Download tomcat tar.gz from archives.
2. do sudo and extract it to /usr/local
3. give ownership to your username by chown -R username foldername
4. set up startup and stop scripts.
5. Access via localhost:8080 to check installation.

Tutorials & Help

There would be times when we forget things which we have been using for a long time. Then you would want to go to basic tutorial and read all about it. There is no good re-writing the commands and explanations here in blog but it is good to have links to the tutorials and help documents. This post will contain links to well-written help documentation for everything I will need. Please contribute to it, if you wish.

1. JAR basics - Documentation from Oracle
   
2. Tutorial on Marathon testing tool
3. How to install Eclipse plugins - manually and the easy way?
   
4. How to install Kindle for Linux
   

Ubuntu: Terminal display help

Here are the few tricks up your sleeve for utilizing the scripts to customize the terminal display.

1. To have the shell prompt always start at the left hand side of terminal:

When you have long pathname for the current working directory and you have a even longer command to execute, the display gets all screwed up. So it always helps you to have the shell prompt start at left hand side of terminal. You have to change the PS1 - to customize your bash prompt.

In the fig, you can't distinguish where the dir name ends and where the command starts. So open ".bashrc" file in home folder

gedit ~/.bashrc

and add the following code. Note: as a good coding practice, add a line of comment saying when and why added the piece of code and add the new code at the end of the file.
PS1=$'\[\e]2; \h::\]$PWD\[\a\]\[\e]1;\]$(basename "$(dirname "$PWD")")/\W\[\a\] \w\n\T \$ '

Then the bash prompt becomes like this,
The current working directory name in first line(s) and in a new line current time followed by $ for you to type in the command.

Much better huh, what exactly does the code do? Let's review from back
\$ - if the effective UID is 0, a #, otherwise a $
\T - the current time in 12-hour HH:MM:SS format
\n - newline
\w - current working directory with ~ instead of $HOME directory
\[ \] - begin and end of a series of non-printing characters
\W - basename of the current working directory with ~ instead of $HOME directory
\e - escape character
\h - hostname

Source: to customize you bash prompt you can follow this help document.

You can also create aliases for opening and refreshing bashrc file like this (if you have to change it frequently)

In "~/.bashrc" file add the following code,

alias refprofile=' source ~/.bashrc'
alias editprofile='vi ~/.bashrc'

whereas, editprofile - opens the file for editing and refprofile - refreshes the file to reflect the change made.

Happy customization!

Ubuntu: Set up iSCSI initiator and target

There will be a lot of posts about iSCSI hereafter, as I am working on the SAN protocol for my graduate project. This was the source I used : http://www.howtoforge.com/using-iscsi-on-ubuntu-9.04-initiator-and-target

This is what I did:
Follow the set up till creating logical volume.
If your system got a partition (/dev/sdan - where n is any number between 1 and 9) which is free and doesn’t contain a partition table(no data or os in it). Use Gparted to format the disk to any type. Then follow the steps below:

sudo fdisk -l(lower case L) - This will list the device partitions size and ID

pvcreate /dev/sda3 - I’m using this partition - On success it would say “Physical volume /dev/sda3 successfully created

vgcreate vg0 /dev/sda3 - On success “Volume group vg0 successfully created”. vg0 - name of the volume group. Any format is fine.< lvcreate -L20G -n storage_lun1 vg0 - From the source 3. After creating the logical volume, follow the source instructions. At the place of iqn.2001-04.com.example:storage.lun1 192.168.0.100 substitute your IP address. I’m going to have both target and initiator at the same computer for now. So I’m using 127.0.0.1 in place of 192.168.0.100 in all forthcoming commands. However, 3260 port number is same and it is default for iSCSI.

------------------------------------------------------------------------------------------------------------------------------------
Now the above post is good but once I partition the sdb disk and changed some file I had problem logging to target. This is the error I got.

iscsiadm: Could not login to [iface: default, target: iqn.2001-04.com.example:storage.disk2.amiens.sys1.xyz, portal: 192.168.1.101,3260]: iscsiadm: initiator reported error (15 - already exists) iscsiadm: Could not log into all portals. Err 15.

I tried couple of solutions like deleting the devices modules and taking a look at the /var/log/messages. Then I jumped at this page and all I had to install tgtadm service and it was piece of cake from there and simple solution to create small portions to act as targets. Since our project is about able to replay the iscsi event with session details as original event for problem diagnosis, for testing during development we need to create small partitions. I would keep updating about this project. If you need more details, do drop me an email.

Ubuntu: How to create full system backup

Since I'm playing around my computer to install iSCSI target and initiator, I needed to create volume groups. That involves partitioning and formatting of my hard drive. Hence I'm very sure this would result in the loss of files from my hard drive :) So I wanted to take full system back up of my Ubuntu. I found a very useful and simple to-do list here. I am going to list those steps here FMR(For My Reference).

The general idea in taking full system back up in Ubuntu(or any Linux for that matter), you can create one compressed archive of all the files with folder structure,then uncompress the archive when you want and viola you have saved your files! Unlike Windows, the creating back up and restoring can be done when the system is running. Warning: while restoring we have to be careful where we uncompress the files to avoid losing data. Here are the steps:

• Become a root user using "sudo su". Warning: be cautious when you are a super user and execute a command from online, there is a single command which can destroy the entire filesytem!
• Go to root of the filesystem using "cd /". This would be the place where your back up archive file would be created. Feel free to create the file in remote drives directly.
• Then use the following single command to create the backup archive
  

  tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

  
  This file would be really big and if you want to use better compression technique, you can also create bzip2 files.
  

  tar cvpjf backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /

  
  This commands take a long time to create the file. The explanation for the command and the options:
  • c- compress, v- verbose, p- preserve permissions
    
  • backup.tar - filename for the backup file
  • You need to exclude the directories that doesn't contain useful files and the back up file itself. Remove any /mnt if you need to avoid backing up other mounted partitions too.
    
  • Ignore the error message like "tar: /: file changed as we read it
    tar: Exiting with failure status due to previous errors" which you might get at the end of the process.
  
  
• Restoring:
  The command to restore the file from the same place as root of the filesystem,
  

   tar xvpfz backup.tgz -C /

  
  Make sure you create the folders that you excluded from the back up archive. So this would be the list of directories if you had used the above command:

  mkdir proc
  mkdir lost+found
  mkdir mnt
  mkdir sys
  etc...

  
  Then reboot the system to complete the process.

Thanks to the source: http://ubuntuforums.org/showthread.php?t=35087

Ubuntu: Configuring Webcam and Microphone in Lenovo Y530

I have a Lenovo Y530 laptop. I had Windows then being the devoted Linux girl, I installed Ubuntu 9.10 on it. The laptop has built-in webcam(1.3M pixels). It was working fine with windows but got screwed up in ubuntu. So I set out to find out how to fix it. At first glance it looked like there was no proper solution available online. But after hours of search, I found the easy steps buried under a bunch of useless/complex solutions. So I wanted to document the exact steps I followed to configure the webcam.

• I tried installing cheese(webcam booth tool for linux) and used Ekiga softphone. You can use either of this tool to check if the webcam is listed in video devices list. In Cheese, the screen had this test screen and both the tools din't recognize the webcam.
• Then in Terminal, try "lsusb". It would list all the USB devices connected to the computer. A sample lsusb would look like this (this has no webcam)
  

  
  Bus 005 Device 002: ID 0483:2016 SGS Thomson Microelectronics Fingerprint Reader
  Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
  Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
  Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
  Bus 002 Device 003: ID 046d:c016 Logitech, Inc. M-UV69a/HP M-UV96 Optical Wheel Mouse
  Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
  Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  

  The 8 digit number followed by ID is the unique ID of the hardware device. If at all you want to search for solutions for any device use this number in your searches and you will more likely land in the page you want. When I ran this cmd, the output din't have any device listed in it. Then in some thread in Ubuntu forums(an excellent place to start your search), someone asked to use hot keys to toggle the webcam. In Lenovo Y530, its Fn+Esc. After that, lsusb listed the device as in,

  Bus 002 Device 002: ID 04f2:b105 Chicony Electronics Co., Ltd

• Now, I searched the ID with all related keywords, it landed here. I checked for my device and saw that it was complaint and all i have to do is download the v4l dvb drivers and viola.
• Then follow the steps listed in this link. I downloaded the latest version of V4L/DVB via web browser and following cmds "make", "make install". [don't forget to do, "sudo make install" - I know its simple but trust me you don't want to miss it]
  

Then happened the weirdest thing. After rebooting the system and checking if webcam worked with cheese, the microphone got screwed. When open the sound preferences dialog(System->Preferences->Sound), in the Input tab, there was no device listed. Without proper knowledge about the thread flow, I removed pulseaudio from the system. Then it was a fresh start as with the webcam. The steps followed were the following:

• In Terminal, use this cmd

  lspci -v | grep -i audio

  
  The result would print out the Audio device if configured right. In my case, it returned empty result.
• So I downloaded the alsa drivers, util, lib files from here. Followed this steps
  • copy the tar.gz files to "/usr/src/alsa" - if you have the folder already or create one if you don't
  • Then unzip the files using "sudo tar xjf alsa-driver*". Similarly util and lib files(change tar options as you require)
  • Then we have to go each directory and do "./configure", "make", "make install".
  • During the "./configure" if any error pops, you have to clear that for the installation to complete. I am just listing the problems I faced.
    • Problem: During the "make" of Util files
      

       xmlto man alsactl_init.xml
      /bin/bash: xmlto: command not found
      

      
      Solution:

      sudo apt-get install xmlto

    • Problem: During the "./configure" of Util files

      checking for new_panel in -lpanelw... no
      configure: error: panelw library not found

      
      Solution: Create soft links to the library files. click here for source

       sudo ln -s libpanelw.so.5 /usr/lib/libpanelw.so
      sudo ln -s libformw.so.5 /usr/lib/libformw.so
      sudo ln -s libmenuw.so.5 /usr/lib/libmenuw.so
      sudo ln -s libncursesw.so.5 /lib/libncursesw.so

    
    

Then reboot the system. Now everything works perfectly fine.

Later added: As I mentioned in the post, I accidentally removed pulseaudio from my system. So when I tried to do skype call, both sides webcam worked fine. But audio had too much static noise. So I had to install pulseaudio. Then reinstalled skype. Skype call works fine. But still if I use the system's music player, then try skype calls again audio has static noise. I read from a forum that "sudo killall pulseaudio" and then restarting skype helps. But that is not efficient solution. If someone has permanent solution to this please drop a comment. Thanks in advance.

Servlets

Servlets are replacing CGI script at most enterprise computing. They both perform similar operations of creating and displaying HTML pages dynamically. The basic differences between the two:

• CGI - Platform dependent, hence they cannot utilize server's capabilities; spawns new process for every request, hence resource intensive;
• Servlets - Since written in Java they can be executed in servers without any modifications as per "write once, run anywhere"; spawns new threads within the server process, hence lite weight.

Lifecycle of servlets:

• init() - initializes the servlet
• service(ServletRequest , ServletResponse) - all url's access this method and here we delegate to other service methods of form doXXX(HttpServletRequest , HttpServletResponse)
• destroy() - destroys the servlet

Reset root password for phpmyadmin/mysql

I recently faced an issue where I forgot the root password for mysql and ended up searching for simple procedure to unlock it by resetting the password. So I want to save here the steps to unlock the phpmyadmin if I should loose my root password again. Then I can skip the searching-for-an-hour part and go straight to the unlocking part. (Thanks to fredanthony and for source thread: click here)

The error message you get when you enter wrong password in the login page is :

"#1045 - Access denied for user 'root'@'localhost' (using password: NO) "

Steps:

1. Stop mysql - $ /etc/init.d/mysql stop
2. Kill all mysql related processes. The ps list can be pretty big and confusing so better use grep to filter mysql processes and use only those pids in kill command. - $ ps waux | grep mysql, $kill -9 pids
3. Open a new session of mysql by skipping permissions - $ /usr/bin/mysqld_safe --skip-grant-tables
4. While the above session is open, in a new tab open a new session of mysql - $ /usr/bin/mysql
5. Inside the new session,change the database - (inside mysql)$use mysql;
6. The critical part of our unlocking process is resetting the password for root user -(inside mysql) $UPDATE user SET Password=PASSWORD('YOUR_PASSWORD_HERE')
   WHERE Host='localhost' AND User='root'; - This step sets new password for the root user.
7. Then quit the mysql sessions and restart the mysql - $quit, $/etc/init.d/mysql restart
8. try logging into phpmyadmin with new password you set in update command and viola!
   

P.S: List of useful SQL command syntax - source

METHOD 2:
This step is to change the MYSQL root password. Source: Ubuntu help documentation. In a terminal enter the following command:

"sudo dpkg-reconfigure mysql-server-5.1"

However verify you have mysql-server-5.1 in your system. To find that, in terminal type "mysql -V". The result would print out details about the mysql server installed in your system.Sample result was
"mysql Ver 14.14 Distrib 5.1.41, for debian-linux-gnu (i486) using readline 6.1". Hence I use 5.1 in the above command. When you run the reconfigure command, it will prompt you to enter password and restarts the mysql daemon.

Ubuntu: Tips and tricks

An excellent and powerful feature about Ubuntu or any other Linux machines is that you can do anything/everything with a single command in a terminal. As with great powers comes great responsibility of understanding(possible) and remembering(absolutely impossible) those powerful commands! So this post would have simple and sometimes complex(which I doubt) commands for working your way around Ubuntu. Please contribute to this list, if you wish. Thanks in advance.

commands:
/tr>

Install eclipse	sudo apt-get install eclipse
Uninstall eclipse	sudo apt-get autoremove eclipse
Copy entire directory in Ubuntu	cp -r src_path/folder/* dest_path
Install LAMP stack	sudo apt-get install lamp-server^ - ^ cadet symbol is mandatory
Restart/Start/Stop Apache	sudo /etc/init.d/apache2 restart/start/stop
Install phpmyadmin	sudo apt-get install libapache2-mod-auth-mysql phpmyadmin
Cleaning up after any uninstall	sudo apt-get autoremove
Start/Stop MySql	/etc/init.d/mysql start/stop
Create softlink for phpmyadmin in Ubuntu 10.04(access as http://localhost/pma/index.php)	sudo ln -s /usr/share/phpmyadmin /var/www/pma
Open folder as root user - AVOID this and prefer using terminal and sudo for executing commands	gksudo nautilus
Print out processor information	cat/proc/cpuinfo
To find out if 32 bit or 64 bit linux kernel	uname -a - if result has i386/i486/i586/i686 GNU/LINUX, then 32 bit or x86_64 GNU/Linux, then 64 bit
To view .chm files in Ubuntu - chm = Microsoft compiled HTML files format	Just to read use "xchm"(sudo apt-get install xchm). To edit the file and republish in different format, follow the source here.
To find the version of JDK in your linux	"java -version"
To create aliases for often used complex commands	Cmd: alias alias_name="alias_cmd --options".If you got more aliases and you need it for all sessions hereafter, create a file "./alias" (name is for your reference, any name should do) in home directory. Add all the aliases command you need.Add the following lines in file $HOME/.bashrc, # Source the aliases, if a separate file exists if [ -e $HOME/.alias ]; then [ -n "$PS1" ] && . $HOME/.alias fi and execute "source $HOME/.bashrc". Source here.
To change password for a user account	cmd: "passwd". You will be prompted to enter current, new and retype new password. It should be more than 6 char long.
To change login shell	Find the current shell, "echo $SHELL", the env variable has the current shell info. Then find list of valid shell logins for your system by "cat /etc/shells". Finally to change to new shell, "chsh -s newshell" where newshell is the full pathname taken from /etc/shells. You will be prompted for password. It will take effect from next login.
To clear the contents of a file without deleting it	We know "cat filename" - print the file on terminal, "nano filename" - to edit line by line, "> filename" - clear the contents of the file without destroying the file.
To use gcc-3.x version instead of default gcc in Ubuntu in 10.04	You need to download required 3.x version, I need 3.4 so I downloaded gcc-3.4-base, cpp-3.4, and gcc-3.4 Then "sudo dpkg -i *.deb", this would select the removed databases and install the gcc. Your default gcc would still be the same and both version co-exist. While compiling just use "gcc-3.4". Note: "gcc -v" displays the version of gcc.
To execute user defined script from any directory in any session	Lets say you got a script named "first" and you want to execute the script just like other linux commands like "echo" from anywhere at shell prompt. In short, you need to add your bin directory to the PATH env variable. Thats what exactly we do here: 1. "mkdir bin" --> in your home directory create a directory called bin . Its a good practice to use "bin" folder to store your scripts. 2. "cp first ~/bin" --> copy all your tested script files to the bin directory. 3. "cd" --> go to root directory, thats where you can find .bash_profile. 4. "vi .bash_profile" --> open the profile in vi editor. This file has all startup settings. 5. "export PATH=$PATH:~/bin" -->append the path of bin directory to env variable PATH so system can find the executable. Don't forget the ":" and replace "~" with full path to your directory if you are using other directory than your home. Save the vi file using ":wq". 4. "first" --> it should work from any directory of the user.
How to install apache server	In terminal: "sudo apt-get install apache2". This would install,configure the server for you. To check in browser, "http://localhost"
How to open "rar"/"zip" files	1. First install "sudo apt-get install unrar". Then "unrar x filename.rar" where filename.rar = your file2. To unzip files, in terminal "unzip filename". -d option specifies the target directory for the files.
Create alias ip address in Linux	"ifconfig INTERFACE:ctr IP_ADDR netmask 255.255.255.0 up"Exp: INTERFACE - the interface in which you want to add the new ip address. ctr - start from 0 and increment for every new alias ip. IP_ADDR - the new alias ip address you want to add. netmask, up - keywords mandatory. Calculate the netmask and update it. If i want to add 192.168.30.10 to eth1 the command would be "ifconfig eth1:0 192.168.30.10 netmask 255.255.255.0 up".
To display unix timestamp	"date" displays the date and time. If you want the unix timestamp use "date +%s". Eg: For Mon Jan 24 21:42:57 PST 2011, the timestamp is 1295934173. Use this site to find out timestamp given date and time.
How to find version of Tomcat	1. "cd $TOMCAT_HOME/bin" -->Go to TOMCAT_HOME - root folder of tomcat installation 2. "sh version.sh" -->will print out the version details of tomcat, jdk
How to open new terminal and execute commands from a terminal	The command is "screen". In the old terminal, "screen command_to_execute" will open up terminal inside this one. Look here for options to detach the new terminal.

P.S: Usually blogger has a bugging problem of inserting unwanted spaces if you insert a table in Edit Html section. To avoid the pitfall, after pasting the table code, remove all the spaces. Hence the code like this (avoiding printing it as table)

table
tbody
tr
td /td
tr
/tbody
table

After removing the extra spaces would look like this,

table tbody tr td /td /tr /tbody /table

Now the code looks confusing, so better edit the table with spaces and finally before publishing it remove the spaces. Hope this helps someone!

To-Do: Run a marathon

Just like every other Sunday morning, I started the hike in Mission peak. The trial never gets easy even if you hike them every week. The climate was awesome as usual. While we were climbing down the trail we saw the runners of 23rd Annual Ohlone 50K Wilderness Run

Its a 31 mile run for some good cause. I saw people from all age groups run for the cause. When I saw them, I added a new item to my bucket list[list of things I plan to do before life ends] and that is to run a marathon for a cause. And I really do hope I am able to do it not just once but almost every year. I will make it happen soon!! For now I will just add my small achievement of touching the mission peak here.

View from Mission peak

Looking back at the trail

PHP MySql POST GET Cookies Session - Bits and information

I always feel its easier to remember details if we learn it as differences between two objects. So I want to blog the basic differences between cookies and session, POST and GET method. And bits and pieces of information about UNION,UNION ALL and some more. This post has so many random information. Let me know if you find something wrong or different.

Three ways to move information across web pages:

1. links: a link with href set to destination page.
   
2. Forms: set action to point to target, method of how server should process information sent, encoding type optional. Both 1 and 2 requires user action like a click.
   
3. Without any user action, user header in PHP to move to another page. header("Location: page.php");
   

Cookies:

• Cookies are stored at the client's computer.
• Once stored at client comp by an application, it can be accessed by any web page of that application.
• It contains variable = value pairs of information.
• Cookies are more useful for applications that has no database.
• Cookies will be available from the next page and not in the current page where it was set.
• In PHP, setcookie('variable','value'); sets the cookie and if you don't mention time, this cookie will expire when user exits the application.
• On setting the cookie, it gets stored in one of many built in arrays of PHP called $_COOKIE. For accessing your cookie variable just refer the array with the variable name like $_COOKIE['variable']
• If we want our cookie to be present even after user exits application then we have to mention the expiration time while we set the cookie. We can use two functions along with setcookie() - time and mktime. time() - returns the current time, we have to add the seconds for which the cookie needs to be alive. eg. setcookie('var','val',time()+3600) - to set cookie alive for 1 hour(60*60 seconds). mktime() - returns date and time and the order of arguments passed is hr,min,sec,mon,day, year. eg. setcookie('var','val',mktime(3,0,0,5,6,2010)); - to make it expire by 3am of May 6th 2010.
  
• To remove cookies, use the same setcookie without any values passed. eg. setcookie('var'); setcookie('var','');
  
• setcookie limitation: It has to be before any output is sent to the client browser. This limitation is with both session and cookie.
  

Session:

• Session details are stored in a file at the server side. In Unix and Linux systems, in \tmp folder and in Windows folder called sessiondata. We can change the location of where the file is stored by chaning session.save_path value in php.ini.
  
• Session ID - long nonsense number for every client which cannot be guessed or forged. In PHP, system variable is used for session ID - PHPSESSID.
• Session ID is passed to every page to access the session details. There are three ways to pass them. 1. If cookies turned on, use the cookies. 2. For links, use the URL. 3. Use hidden variables for form with POST method.
• Session variable is got and stored in $_SESSION built in array(similar to $_COOKIE array in cookies). They are accessed same way as in cookies.
• For session to work we need to enable, track_vars while installing PHP.(default its turned on from PHP 4.0).
• If cookie is turned off at the client side, trans_sid should be enabled to transfer session id. To enable that use session.use_trans_sid = 1 in php.ini.
  
• Start a session: session_start() - If sessionID is found, then load $_SESSION with variables and their values. If no ID is found, then it is first time so create a new session and set PHPSESSID.
• Save a session variable: $_SESSION['var'] = 'val';
• Close a session: session_destroy() - destroys the session details.
• session_id() - returns the PHPSESSID value - current session id.
• unset($_SESSION) - unset the session details in the current page.
• Limitation: session has to be set before any output is sent to the client browser.
  

We know that POST and GET are ways of indicating how the server should process the information sent by the form.

POST:

• Sends information in 2 steps. 1. browser contacts form processing server specified in action. 2. Once contact has been established, send data to server in separate transmission.
• On server side: 1. read parameters from a standard location. 2. After read, decode parameters before application use form variables.
• To get post variables from earlier form use built in array $_POST.
  

GET:

• In a single transmission, data is sent to the server. Data is appended by the browser to action URL.
• This is the default method if not specified otherwise.
  
• On server side, gets information passed at the end of the URL.

POST or GET what to use?

• GET: best transmission performance(single transmission) and apt for small forms with short/few fields.
• POST: Apt for forms with many/long text fields.
• If inexperienced with server programming use GET to avoid extra steps of processing- read/decode as in POST.
• For security purpose, use POST to avoid the information you transmit to be available in open for hackers to track. POST has security holes too but atleast it has encoding when transmitting.
• To invoke server processing outside form tag, eg in a tag, use GET because it lets us use form-like parameters as part of URL.
  

ENCTYPE field in FORM tag:

• Two types of enctype options available. 1. Multipart and 2. Text/plain
• Multipart: forms with file selection fields for upload by user.
• text/plain: used along with mailto in action attr of form tag. While sending forms to email server rather than a server.
• The default encoding type: Internet media type Application/x-www-form-urlencode.

UNION and UNION ALL:

• Union all - combines rows from multiple row sources into one result set. It includes duplicates.
• Union - does the same thing as union all but excludes duplicates, result would be sorted here without duplicates.

Polymorphism in OO langauges:

• Ability of 2 or more objects belonging to different classes to respond to exactly same message in different class specific ways.
  

Thread programming in Java

I spent a day in library refreshing stuffs I learned long back(not so long!) while I was in undergrad. Since my memory is fresh with these topics, I thought I should blog them so next time I can find it all in one place. On a different note, a day spent studying/reading in library is well spent! Okay back to Threads in Java.

Thread - class, Runnable - interface which has the thread implementations. For our application class to use thread we need to either implement or subclass or make the class a member class of our application. Thus three ways to create threads:

1. Extend Thread class: For standalone applications that don't need to extend another class.
2. Implement Runnable interface: For classes that need to extend other classes. Due to single inheritance, we can achieve threads by implementing the interface. Eg: Applets that extend JApplet and can't extend Thread too.
   
3. Construct a thread passing an inner class that is Runnable: For threads with tiny run methods and little outside interaction.

Thread life cycle functions:

• Thread() - When subclassing Thread class
• Thread(name) - when subclassing + for debugging
• start() - Begins process of assigning CPU time to a thread. Finally it results in calling run() method of the thread object.
• suspend(), resume(),stop() - deprecated. Because these suspend and resume functions can cause deadlocks.
• interrupt() - interrupt whatever the thread is doing
• boolean isAlive() - returns true if thread is neither finished or terminated by call to its stop()
• join() - to wait for a thread to finish
• get/setPriority() - higher priority threads gets first chance of CPU
• wait(), notify(), notifyAll() - semaphore and for thread synchronization.
• synchronised - keyword - can be used for any object in Java. When used over an object or a method, threads will wait if one thread has already started running.

Thread synchronization:
The wait, notify and notifyAll methods allow any java.lang.Object to be used as a synchronization target.

wait():
This causes the current thread to block in given object until awakened by notify or notifyAll.

notify():
Randomly selected thread waiting on this object is awakened. Then that thread tries to regain the monitor lock. If wrong thread is awakened, will result in deadlock.

notifyAll():
All threads waiting on this object is awakened. Then all try for the monitor lock. Hopefully one of them succeed.

Best practices:

• To avoid deadlock, use notifyAll to wake up waiting threads.
• To avoid using deprecated method - stop, use the variable as a flag and use it to find if the thread is done executing.